{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Music Recommendation的数据集——数据探索&特征工程\n",
    "\n",
    "数据集包含5个主要文件：\n",
    "1.train.csv: 训练数据(用户点击歌曲表)\n",
    "    msno: 用户id，加密String\n",
    "    song_id: song id，歌曲id\n",
    "    source_system_tab: 触发事件的类型/tab，用于表示app的功能类型\n",
    "    source_screen_name: 用户看到的布局的名字（name of the layout）\n",
    "    source_type: 用户在app上播放音乐的入口的类型\n",
    "    target: 标签。1表示用户在第一次听音乐后会在一个月内继续订阅，0表示没有订阅。\n",
    "2.test.csv: 测试数据(用户点击歌曲表)\n",
    "    id:主关键字\n",
    "    msno: 用户id，加密String\n",
    "    song_id: song id，歌曲id\n",
    "    source_system_tab: 触发事件的类型/tab，用于表示app的功能类型\n",
    "    source_screen_name: 用户看到的布局的名字（name of the layout）\n",
    "    source_type: 用户在app上播放音乐的入口的类型\n",
    "    \n",
    "用户歌曲的原数据\n",
    "3.用户表：members.csv 7个字段：\n",
    "    1）msno：用户ID \n",
    "    2）city：城市 \n",
    "    3）bd：年龄 注意：年龄数据有离群点\n",
    "    4）gender：性别\n",
    "    5）registered_via: 注册方式\n",
    "    6）registration_init_time: 注册时间，格式为%Y%m%d\n",
    "    7）expiration_date: 到期时间，格式为 %Y%m%d\n",
    "4.歌曲表：songs.csv 7个字段\n",
    "     1)song_id：歌曲id\n",
    "     2)song_length: 单位为ms\n",
    "     3)genre_ids: genre 类别. 可多选，用 “|“隔开\n",
    "     4)artist_name：歌手\n",
    "     5)composer：作曲\n",
    "     6)lyricist：作词\n",
    "     7)language：语言\n",
    "5.歌曲额外的信息表 song_extra_infos.csv 2个字段\n",
    "   1)song_id：歌曲id\n",
    "   2)song name ：歌曲名字\n",
    "   3)isrc – 国际标准音像制品编码(International Standard Recording Code )。理论上可用于歌曲id，但产生的ISR没有经过官方授权。因此ISRC中的信息，如国家代码和参考年份可能不正确。且多首歌曲可能共享共一个ISRC，因为一首歌曲的音像制可发行多次。\n",
    "   \n",
    "   项目目标：训练数据由2017年2月服务到期的用户构成，target标签代表用户在2017年3月是否续订了业务。测试集中的数据由2017年3月内将到期的用户构成，需要预测用户是否在到期后的一个月内即2017年4月预定、流失的概率。（打算选用二类分类模型解决）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import lightgbm as lgbm\n",
    "import datetime\n",
    "import math\n",
    "import gc\n",
    "from subprocess import check_output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取所有数据 特征类型为category\n",
    "data_path = './data/'\n",
    "train = pd.read_csv(data_path + 'train.csv', dtype={'msno' : 'category',\n",
    "                                                'source_system_tab' : 'category',\n",
    "                                                  'source_screen_name' : 'category',\n",
    "                                                  'source_type' : 'category',\n",
    "                                                  'target' : np.uint8,\n",
    "                                                  'song_id' : 'category'})\n",
    "test = pd.read_csv(data_path + 'test.csv', dtype={'msno' : 'category',\n",
    "                                                'source_system_tab' : 'category',\n",
    "                                                'source_screen_name' : 'category',\n",
    "                                                'source_type' : 'category',\n",
    "                                                'song_id' : 'category'})\n",
    "songs = pd.read_csv(data_path + 'songs.csv',dtype={'genre_ids': 'category',\n",
    "                                                  'language' : 'category',\n",
    "                                                  'artist_name' : 'category',\n",
    "                                                  'composer' : 'category',\n",
    "                                                  'lyricist' : 'category',\n",
    "                                                  'song_id' : 'category'})\n",
    "members = pd.read_csv(data_path + 'members.csv',dtype={'city' : 'category',\n",
    "                                                      'bd' : np.uint8,\n",
    "                                                      'gender' : 'category',\n",
    "                                                      'registered_via' : 'category'},\n",
    "                     parse_dates=['registration_init_time','expiration_date'])\n",
    "songs_extra = pd.read_csv(data_path + 'song_extra_info.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看空值形况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "msno                       0\n",
       "song_id                    0\n",
       "source_system_tab      24849\n",
       "source_screen_name    414804\n",
       "source_type            21539\n",
       "target                     0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.apply(lambda x: sum(x.isnull()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id                         0\n",
       "msno                       0\n",
       "song_id                    0\n",
       "source_system_tab       8442\n",
       "source_screen_name    162883\n",
       "source_type             7297\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.apply(lambda x: sum(x.isnull()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可看出train和test的source_system_tab、source_screen_name、source_type三个特征都存在空值情况需要填补"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "song_id              0\n",
       "song_length          0\n",
       "genre_ids        94116\n",
       "artist_name          0\n",
       "composer       1071354\n",
       "lyricist       1945268\n",
       "language             1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "songs.apply(lambda x: sum(x.isnull()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可看出音乐曲库中genre_ids、composer、lyricist三个特征都存在空值情况需要填补"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "msno                          0\n",
       "city                          0\n",
       "bd                            0\n",
       "gender                    19902\n",
       "registered_via                0\n",
       "registration_init_time        0\n",
       "expiration_date               0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "members.apply(lambda x: sum(x.isnull()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可看出用户表中gender特征存在空值情况需要填补"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "song_id         0\n",
       "name            2\n",
       "isrc       136548\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "songs_extra.apply(lambda x: sum(x.isnull()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可看歌曲库扩展信息表中isrc特征存在空值情况需要填补"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看数据类别型分类形况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "msno属性有30755的不同取值，各取值及其出现的次数\n",
      "\n",
      "MXIMDXO0j3UpaT7FvOSGW6Y5zfhlh+xYjTqGoUdMzEE=    5819\n",
      "o+5RNlSWrzvrphgBNGIo1FLkGxBgyICns6qXj3nS7Pk=    5537\n",
      "FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=    5511\n",
      "KGXNZ/H3VxvET/+rGxlrAe7Gpz2eKMXyuSg3xh8Ij1M=    4217\n",
      "cqjRBV/jWN2ujhc+z/4tz+Mj6xEfflAAt6qBXCqxKvw=    4008\n",
      "hYJpPvGod6vy09TnlXdQe3Q0vlxju5u5Ruf8V2XkTio=    3926\n",
      "SZ5NNypqaTWljFO1HiVZwkw3713+rM9x/JNdJd8/fzc=    3733\n",
      "dU4RbzpIRRd/EkA9Xncpy9CglzDBZp7nKMfdnfr3Aj8=    3686\n",
      "OOUnJuX4SteRhUdJZ9B2DqtfiwsfcZVBefEhXLeBsFg=    3239\n",
      "frZtrrN1Y2ZqZX3VFiq7FpAvLth3kZJNovT9cmyn2O4=    3177\n",
      "7xiBI5xU3P2/IiR9teB7jySfzWo96JGikGajteLd3Cs=    3133\n",
      "4tuTIArXO3WO8/zS9y2CC34ywEfChjAKUchlKW00DMM=    3030\n",
      "JkQacE3rvmhh65R04eDLbu+M7MCkpzmHwMGrbZo0puc=    3027\n",
      "gxxBbzV3eE2XGjUrFVB2FzAve55Oe1s86HD+OEh36Gw=    2973\n",
      "1d4acB2bhEZCFjSRwvO4ls8PrBtvNTlkcAcxYx8FcWE=    2930\n",
      "jL7ukei/Kb4gDCfTG4+VhV4TzLaDecjji/aW+GqWvac=    2922\n",
      "y3IT85vrzY2iBxFg6nsh4Cmg+mV3oqR73TPekK7x1Rc=    2895\n",
      "YU6fAgCFgPkaJ1YSW2838KzGTxmBCfgousTO8jwuHYM=    2883\n",
      "TxneOykZ/MOIbQmA/aJ4EUQUIR9r+Egt6TNQu0WgwpA=    2861\n",
      "LThaiVqGGnVTPmTcmwN/LLo4fVb5dzkduzd7s1SgzIA=    2814\n",
      "UTAElx4+rSh2QViTKHQhC40cphX7ZIYafAOMJOJCshY=    2767\n",
      "2+T8pswRgDmvZ6B4esKi3lWZyT4xqeVKmv19J0/y51w=    2763\n",
      "TDHfjwkLrnoorqjTUFtlzSL47Yl1oEBpHM8k7qHrIdI=    2743\n",
      "mGDObQQojFOJfK2rJKcme282huuk0qDxBzWL22/qYbc=    2689\n",
      "FaH6rl69OOVwbsO30jjdjVXD9wJO8pcVZS6LGy5zTJM=    2647\n",
      "HVcWdf8CEo9s6qwt5V7TpoPYJfRlQSTA6b7kxEAImpc=    2629\n",
      "s8Oce1igq03hako2pQCfeXyPVkBs77ArjdrZEQAjpbk=    2619\n",
      "6aH1YAi5YwvZUHtpjEaMQwM0p4wP4XLrkWQSdXDF9L8=    2590\n",
      "T+FJrpYi77CWTUE5nO/vliojRaTPgt2bRUQHy5lhXrQ=    2557\n",
      "mDJCU+fKu/mbdk9l4SmPYiJwpeMyK44o9wmG1X3735A=    2528\n",
      "                                                ... \n",
      "PFrsTH0marxGTR0EbCx8wRJ+JTu7AOD7TK3SxkI68Kw=       1\n",
      "ONH5XAZCS4hGA6b4mNjCHh0KlWmFHoLLyWHV5LfiPNQ=       1\n",
      "z/YOmKw1VqcFsXrriL1GSE4lExCLch1lPbjkKdNrDNc=       1\n",
      "fJ+uM/srOi+HXcu4g5gyy+lxBSsz/Jy31pgjHpN4Iwo=       1\n",
      "RQPYmGuAejl9Eps1mv1clLSyBbe6FUwd9Mk1J3q8lAY=       1\n",
      "uEDRRfx9ctzj6Pfb77Q74OHmnhy2Fa6BuJChxWjkEXQ=       1\n",
      "ap/DeQ7G+WwNwdGRFpEFbJcv6IYw/EC6dRgYeChW258=       1\n",
      "GbglQWtCVBbrLstgf/srM3+6MwcqJvXWCrzM4lzktNQ=       1\n",
      "v/m3MFZDHGsT7B+lEY1BieIr5XUarryS7YiVi2384xM=       1\n",
      "fjBfvjsuGQnWqoD9v31lQunpFcv8dhaYH8ky7oQGy6Q=       1\n",
      "fhAGDOMJkbhMNVWB5kRtBRN2trpLVptkNrTI9eDpmGE=       1\n",
      "fcbYZgWAk5DQT5XNIz39dP7fcoZZK+/wAK4cxUCSpBo=       1\n",
      "U8+k0YmjY2aDIM7Wd9ejIF/XqZkoWbBlv+2booso86A=       1\n",
      "3oAGCnlrLiW96SSxF64jRNb3N+RkHbm77veK9/5vtX8=       1\n",
      "ahgEG1VAVLEcpQ5yL51Uk9Mr0gT17mkhYYcE0OdGxxI=       1\n",
      "6akPA/c/wV48xkGFM5c8uGYuShSDysLudR+5abP13BM=       1\n",
      "RqcQvcdo9CiaMifPj+5SSV4rFHIzCxJy/ZYMUcsKhHs=       1\n",
      "1TknsLZfkNn/eJJIuGOzyWiTDuykew62rlG2qFaVlLI=       1\n",
      "BQfUp7ooAfUi5CC49hq2m7+ahqKJVVmptb3Se+9rr4s=       1\n",
      "4FTP7YbhmUyY5KS8VhEfQW2pa0Xb8gdSinYZmvI46k0=       1\n",
      "y+NGIqN5UKxC3Saa6gl4HErv6hw+UT0PQc+wurTDF+U=       1\n",
      "PuApVUFf/c0t/kB+sxOFZD4ifIY4nlPsmTKyysjQ4KY=       1\n",
      "LQXqFcn7DRZPPCyLIcNpAWE9hGoVi0eA28MyDt1QJoA=       1\n",
      "PcaKGJUq7MqJ6KKidHpI9DNfeximIs73E3S4YyYtvMI=       1\n",
      "0MF7L2iH3CZyScPGkuYocNRqjCDET8/VHEp5UY7TNU0=       1\n",
      "yVZUBQL9It5STse3ZbZsuIw+74GBg43Hi2tLnlqcUak=       1\n",
      "iO3KhnyuWTLKqwTFa7P975wxT3d52wvXVa1wVvcaEhY=       1\n",
      "/QSjtgC130uZXSS7wba5rOKbxUwoXxCMogEklv7gkUs=       1\n",
      "yeowVOwuBlf9/tsxXOKwjI5Ri7ux1o5p5gYh6BAEJJc=       1\n",
      "K2ua9jwAdeJOTJTEJGzYirzdhZa9uV0MiHta8yblMvw=       1\n",
      "Name: msno, Length: 30755, dtype: int64\n",
      "\n",
      "song_id属性有359966的不同取值，各取值及其出现的次数\n",
      "\n",
      "reXuGcEWDDCnL0K3Th//3DFG4S1ACSpJMzA+CFipo1g=    13973\n",
      "T86YHdD4C9JSc274b1IlMkLuNdz4BQRB50fWWE7hx9g=    13293\n",
      "wBTWuHbjdjxnG1lQcbqnK4FddV24rUhuyrYLd9c/hmk=    13079\n",
      "FynUyq0+drmIARmK1JZ/qcjNZ7DKkqTY6/0O0lTzNUI=    12855\n",
      "PgRtmmESVNtWjoZHO5a1r21vIz9sVZmcJJpFCbRa1LI=    12004\n",
      "YN4T/yvvXtYrBVN8KTnieiQohHL3T9fnzUkbLWcgLro=    11835\n",
      "M9rAajz4dYuRhZ7jLvf9RRayVA3os61X/XXHEuW4giA=    11745\n",
      "U9kojfZSKaiWOW94PKh1Riyv/zUWxmBRmv0XInQWLGw=    11521\n",
      "43Qm2YzsP99P5wm37B1JIhezUcQ/1CDjYlQx6rBbz2U=    11131\n",
      "cy10N2j2sdY/X4BDUcMu2Iumfz7pV3tqE5iEaup2yGI=    10791\n",
      "J4qKkLIoW7aYACuTupHLAPZYmRp08en1AEux+GSUzdw=    10565\n",
      "750RprmFfLV0bymtDH88g24pLZGVi5VpBAI300P6UOA=    10500\n",
      "+SstqMwhQPBQFTPBhLKPT642IiBDXzZFwlzsLl4cGXo=     9908\n",
      "IKMFuL0f5Y8c63Hg9BXkeNJjE0z8yf3gMt/tOxF4QNE=     9844\n",
      "v/3onppBGoSpGsWb8iaCIO8eX5+iacbH5a4ZUhT7N54=     9736\n",
      "DLBDZhOoW7zd7GBV99bi92ZXYUS26lzV+jJKbHshP5c=     9244\n",
      "p/yR06j/RQ2J6yGCFL0K+1R06OeG+eXcwxRgOHDo/Tk=     9038\n",
      "Xpjwi8UAE2Vv9PZ6cZnhc58MCtl3cKZEO1sdAkqJ4mo=     8883\n",
      "8Ckw1wek5d6oEsNUoM4P5iag86TaEmyLwdtrckL0Re8=     8851\n",
      "BITuBuNyXQydJcjDL2BUnCu4/IXaJg5IPOuycc/4dtY=     8845\n",
      "n+pMhj/jpCnpiUcSDl4k3i9FJODDddEXmpE48/HczTI=     8775\n",
      "BgqjNqzsyCpEGvxyUmktvHC8WO5+FQO/pQTaZ4broMU=     8725\n",
      "OaEbZ6TJ1NePtNUeEgWsvFLeopkSln9WQu8PBR5B3+A=     8658\n",
      "3VkD5ekIf5duJm1hmYTZlXjyl0zqV8wCzuAh3uocfCg=     8428\n",
      "/70HjygVDhHsKBoV8mmsBg/WduSgs4+Zg6GfzhUQbdk=     8381\n",
      "fEAIgFRWmhXmo6m3ukQeqRksZCcO/7CjkqNckRHiVQo=     8313\n",
      "WL4ipO3Mx9pxd4FMs69ha6o9541+fLeOow67Qkrfnro=     8296\n",
      "fCCmIa0Y5m+MCGbQga31MOLTIqi7ddgXvkjFPmfslGw=     8177\n",
      "VkDBgh89umc9m6uAEfD6LXngetyGhln4vh/ArCGO0nY=     8062\n",
      "L6w2d0w84FjTvFr+BhMfgu7dZAsGiOqUGmvvxIG3gvQ=     7878\n",
      "                                                ...  \n",
      "lqUpSdGHHliQJ4yDj7d4VLF7fIZCzgkMO6SGd0s6eEQ=        1\n",
      "3Hi9DiZcMHk4QurNp7ML5ERr2g+W5g0S7Y+TCwh3eHI=        1\n",
      "lr5nfwRg2fPET/oBf8V1jUbKSf9XnE8iVMRgI/4LjW8=        1\n",
      "lrkMBFJOasA8y60TWi0jbFFLNPp+VrN2KpvNMbTB4Lw=        1\n",
      "3EfRsMJeJUboc9EyIO5ifCMt7JgtBhCAUppzHj5pBYQ=        1\n",
      "lrw17BAvNNoEi6x26YL7YmXFgMMLe2yspECg1qTG8OI=        1\n",
      "lswL7RJzfJeP4/GbrKkFFW2ThSMvemVFtO9XNAkolzc=        1\n",
      "3PhTujIArPdYtLJvbRKJec2TkkatvoCdQNtpzM3hXXA=        1\n",
      "3SERWokvE3m1+qWdTc4v6b4fqJY8ophx69nTNShoRC0=        1\n",
      "lUeU39DuJ0rx/vy1K9o+xEn7a4CgRMVcTczVugJQihM=        1\n",
      "leK97qhSu5Sd270blW5rp3vZHuld9zpwwsKMAS0xQxI=        1\n",
      "lV1HHp0gAPvGyMGahGLUUj8ExwdwpB39mC6YBkTSlv8=        1\n",
      "3jXkOPM0B8GbcTWVlbrKBo/R6UwPIFqcpGGqF57RqY8=        1\n",
      "3gJ7LTS9XnTSqvG1R6bSapM0uH0uS5sO7jTua3bZtZI=        1\n",
      "lWrvce7xOdei2ppTsz8tbL3ftNtc9gG5gEt/KefnO/8=        1\n",
      "3d4PVhCv1IXkvxt+iQqCKkycwuMg2B/ku5N6dzfRs7M=        1\n",
      "lZhH0P67gnegkFYAN1hXPpxBkLLEk7Rag4NMNTrFB5Q=        1\n",
      "3awBp8lQ1deAm9IRqfPkM/O2y/WTev4vi1cHk/30ZMw=        1\n",
      "laGRehDo+Esy5I5tKnazOYIJg4qyY24qXHIgKoL+zeI=        1\n",
      "3WjT2TWoRwUMIhLOt+cBPevgX6Z/VuYeDsUfPla/4jc=        1\n",
      "3SGnTjgep/9n+zdpK6lJyZW+ZsiBTy124ENC85UYwAA=        1\n",
      "3W44tIZsjC2faKA04XlsI0VLHDm1zniXkQVc7MsWMPM=        1\n",
      "lg2Ri4uYngOxVhQhvalE9sqOr6EeYx8HaEb0AJ/lImo=        1\n",
      "lgHgas4mZHR3KBOQ8PE7sB1pihqF/1P2xxdajECbjK8=        1\n",
      "3VMKB95fGSFvc64foEZHnYh9xL5WvxTSBBdKE+6jIDI=        1\n",
      "liqMW+ZmhsoK8PaQ0psC7YRkwk3V/ix556ivS2PPNvA=        1\n",
      "3UI+gLvUawOCDI6hXBPeYYUaTc5yw526IDvXDKTx7OA=        1\n",
      "3U9dXxylpNK4Okhy7vMVouXyUzgsqm/8Y9J83uek8uU=        1\n",
      "ljIAP8A6b4LVeH1cJqWdQpxNjq84PSvf3w98qvPkpWA=        1\n",
      "zz28BZKXDZHb4MpF1Zu/ZvwXJAJVJr6mqlRtJp1RfiY=        1\n",
      "Name: song_id, Length: 359966, dtype: int64\n",
      "\n",
      "source_system_tab属性有9的不同取值，各取值及其出现的次数\n",
      "\n",
      "my library      3684730\n",
      "discover        2179252\n",
      "search           623286\n",
      "radio            476701\n",
      "listen with      212266\n",
      "explore          167949\n",
      "notification       6185\n",
      "settings           2200\n",
      "Name: source_system_tab, dtype: int64\n",
      "\n",
      "source_screen_name属性有21的不同取值，各取值及其出现的次数\n",
      "\n",
      "Local playlist more     3228202\n",
      "Online playlist more    1294689\n",
      "Radio                    474467\n",
      "Album more               420156\n",
      "Search                   298487\n",
      "Artist more              252429\n",
      "Discover Feature         244246\n",
      "Discover Chart           213658\n",
      "Others profile more      201795\n",
      "Discover Genre            82202\n",
      "My library                75980\n",
      "Explore                   72342\n",
      "Unknown                   54170\n",
      "Discover New              15955\n",
      "Search Trends             13632\n",
      "Search Home               13482\n",
      "My library_Search          6451\n",
      "Self profile more           212\n",
      "Concert                      47\n",
      "Payment                      12\n",
      "Name: source_screen_name, dtype: int64\n",
      "\n",
      "source_type属性有13的不同取值，各取值及其出现的次数\n",
      "\n",
      "local-library             2261399\n",
      "online-playlist           1967924\n",
      "local-playlist            1079503\n",
      "radio                      483109\n",
      "album                      477344\n",
      "top-hits-for-artist        423614\n",
      "song                       244722\n",
      "song-based-playlist        210527\n",
      "listen-with                192842\n",
      "topic-article-playlist      11194\n",
      "artist                       3038\n",
      "my-daily-playlist             663\n",
      "Name: source_type, dtype: int64\n",
      "\n",
      "target属性有2的不同取值，各取值及其出现的次数\n",
      "\n",
      "1    3714656\n",
      "0    3662762\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: target, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "for col in list(train.columns):\n",
    "    num_vlaules = len(train[col].unique())\n",
    "    print('\\n%s属性有%d的不同取值，各取值及其出现的次数\\n'% (col,num_vlaules))\n",
    "    print(train[col].value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "id属性有2556790的不同取值，各取值及其出现的次数\n",
      "\n",
      "2047       1\n",
      "2182787    1\n",
      "2164362    1\n",
      "2170505    1\n",
      "2168456    1\n",
      "2190983    1\n",
      "2188934    1\n",
      "2195077    1\n",
      "2193028    1\n",
      "2180738    1\n",
      "2176652    1\n",
      "2186881    1\n",
      "2184832    1\n",
      "2535039    1\n",
      "2532990    1\n",
      "2539133    1\n",
      "2537084    1\n",
      "2526843    1\n",
      "2166411    1\n",
      "2178701    1\n",
      "1859878    1\n",
      "2223767    1\n",
      "2205342    1\n",
      "2211485    1\n",
      "2209436    1\n",
      "2199195    1\n",
      "2197146    1\n",
      "2203289    1\n",
      "2201240    1\n",
      "2221718    1\n",
      "          ..\n",
      "523690     1\n",
      "519592     1\n",
      "298436     1\n",
      "497063     1\n",
      "499110     1\n",
      "492965     1\n",
      "495012     1\n",
      "505251     1\n",
      "507298     1\n",
      "501153     1\n",
      "468401     1\n",
      "474546     1\n",
      "472499     1\n",
      "462260     1\n",
      "308675     1\n",
      "310722     1\n",
      "304577     1\n",
      "306624     1\n",
      "480703     1\n",
      "482750     1\n",
      "476605     1\n",
      "478652     1\n",
      "488891     1\n",
      "490938     1\n",
      "484793     1\n",
      "486840     1\n",
      "464311     1\n",
      "466358     1\n",
      "460213     1\n",
      "0          1\n",
      "Name: id, Length: 2556790, dtype: int64\n",
      "\n",
      "msno属性有25131的不同取值，各取值及其出现的次数\n",
      "\n",
      "KGXNZ/H3VxvET/+rGxlrAe7Gpz2eKMXyuSg3xh8Ij1M=    2489\n",
      "yBXKYfcmQ+TtIEvkLQXabAY2uwudIGn21dEZYAwCDCg=    2389\n",
      "MXIMDXO0j3UpaT7FvOSGW6Y5zfhlh+xYjTqGoUdMzEE=    2075\n",
      "y+PDdTzWxEVgd0r1BH38ib0SuyWzgd1FAV5iMl53Uw0=    2068\n",
      "y3IT85vrzY2iBxFg6nsh4Cmg+mV3oqR73TPekK7x1Rc=    2058\n",
      "o+5RNlSWrzvrphgBNGIo1FLkGxBgyICns6qXj3nS7Pk=    1781\n",
      "FGqBebsinq4fszaFAyN4WqDL6vnhMlMe248eN5EgnWI=    1762\n",
      "mGDObQQojFOJfK2rJKcme282huuk0qDxBzWL22/qYbc=    1675\n",
      "BqnomjS+KJtzWrGRkqLEoO3I7KapwN4B8H9in4Fsf+I=    1629\n",
      "6zb1JIozg37wsF7TsVd/U/ZT3peCtosRl/KpxyZvILs=    1614\n",
      "2pUzjoYYUf9j1xpJOGmzu0Zp+52MdLKsVH0eFLePBbs=    1613\n",
      "Hm3iH5wb4xg6YJa4HKRhTN2OfOecobNmHfxCAl4T374=    1586\n",
      "FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=    1564\n",
      "8G9kFHqzaGUu0BRbHV87yYhrWKEaGkuE6beClg2buAc=    1559\n",
      "K1BJCDeN15EDF2kmNJ4pVKNp6kAhNOyiNOdivzOVe+Q=    1540\n",
      "x1staqrE1JMIUlvyEVgMYgDfZhlTGfpA3qtPSj+N77g=    1480\n",
      "cqjRBV/jWN2ujhc+z/4tz+Mj6xEfflAAt6qBXCqxKvw=    1438\n",
      "pForpfrz+EtnctFemGiaGXRER8TXx3C63pD13pGXwPo=    1434\n",
      "38LUs5mVWNKfyjL7ilf9JCnWZLjnno2obLXr69JJKcw=    1396\n",
      "TxneOykZ/MOIbQmA/aJ4EUQUIR9r+Egt6TNQu0WgwpA=    1395\n",
      "cNssc+2L1Fx77CzASou28E9qfWeJjQKZtfoADkl/xkQ=    1383\n",
      "INZ9nLYzLBH2w7LHo42K0XfM2xe4Z8f1QAPsTQAdWQg=    1367\n",
      "7xiBI5xU3P2/IiR9teB7jySfzWo96JGikGajteLd3Cs=    1349\n",
      "Mpt2fpn7GsL2m33gY79d8pdglz4oYt2uXJVcZmG1xq8=    1320\n",
      "wcpspaw6+dFR9R2y0T9oUSwmB+ZS2Fi8U3KCnbSdFT4=    1319\n",
      "jL7ukei/Kb4gDCfTG4+VhV4TzLaDecjji/aW+GqWvac=    1263\n",
      "l8r5s1W/iTPlGRxMXJDl0aZAk9suTgtodRrbXN8Xly4=    1209\n",
      "saNgxsUQwx4IXBRgFF0w3uJN6KzvQBEv3dI7dDaz97g=    1192\n",
      "2+T8pswRgDmvZ6B4esKi3lWZyT4xqeVKmv19J0/y51w=    1174\n",
      "y2XX7JrDw7iMqTkoHtxZuJ+FYVystzaa7XZ9cAj3jQQ=    1160\n",
      "                                                ... \n",
      "pIIy6axZ+OP7XPM7b553Zy+HyMqSkeKFUrF6Lgi05W0=       1\n",
      "5+xtVmzQs8jZucmazqL+rT7qwAxPJfkiRiR1qwhbqmg=       1\n",
      "amO5aI+iw8XgVKoMf3A7JZ/DZJKjZR8kXLxmGeQgFCE=       1\n",
      "ogEBMZiOPXqckx1F3Je4xj6niEQ5BqcQR0DYprAm5TU=       1\n",
      "pmzRy3WzzPpe1oXoo6mOR9oOscDIgudAm6QGxtv84yY=       1\n",
      "8nYVam9pivEVSZ2So5dNWYfR/2loOl7zdbq9t++XEZ8=       1\n",
      "RNFUrQzf8Aybj+4b7lOVGJgFAYv3hTgX5R5Kd+vgryg=       1\n",
      "70yvcdJEYQhZ0SNpNG1X84iQLlFto3pxr91+NHqJ3sc=       1\n",
      "6twCrThgJVco+vU8wm7t2YcAQsLxp5aoVmuYRzTX8BI=       1\n",
      "eA5vCLWced6ANIAafQaRo8ho7idpnbloVYSbzHzRoT8=       1\n",
      "eCBBK6Xkf/sO0MA6+JumbiNm6fg+1rFhlwRt1MWjDyo=       1\n",
      "QbnZCFL4SlFpGCf1y0W8jLrhtIE8OnDWeOKzWUsPsKQ=       1\n",
      "CY00BwK4N5pOv+spt+fjxy5WHTxfWrHt1MOJql2TKXw=       1\n",
      "KMl6K00dJsFvlqayIsZpLGieaNUzEtumxvgN0I3eS0Q=       1\n",
      "3kKjm2NPnRZqKDg7BLHwOKEENznakPaIHBOMhr4BGGY=       1\n",
      "TyqO3Eiy90EJfaK+b0lxYe7qJOPJG0cGnf44Zr2MBTs=       1\n",
      "hJumhQoGEXyMw9KKgRRd7frsz81EUQcg5b7VYsR55hA=       1\n",
      "UPmYSBGeC0eNbZKe42P+WFBglbqg4I4XYu7CpMLuMig=       1\n",
      "j80eBDtXD1zIhD/nlh5sShtkCdQQldRJRl1sbJdjr9Y=       1\n",
      "oSsDz4WYWLBdCGffA9YXwaQYBySqfCSufC4m28tkaF0=       1\n",
      "jJ6h6y8FjCXMMzZ06Lrh8di8e1oH9a4MlTYJkaX7pTQ=       1\n",
      "3lC72pSCoQvZFSaIl+ORPUYgCOwTsIa2+9SlHYLRwe0=       1\n",
      "CgXjvyM2hYjkP0X69c40SiCbNLh5PgOXxTw7ZW8idMw=       1\n",
      "/rfpzWLnNzZLAU6D8FML+gK8Igqr9JkzfT+vkg4V8eQ=       1\n",
      "r/e9N4Y6OyeCDFLbsHMV64ouHI2oOmCfAJ1ZbhN9pug=       1\n",
      "W7TcIL5S9URhMHc7xZD4MyE1ILo29/hfLevxgIp/Bjg=       1\n",
      "qzBE+D+8tZjIJs/vexZQbHPxFF3usJBaYX7/K+XhrpU=       1\n",
      "WSktBowbANZVhMcOp2gv/UAsLia3tI58Mshs7l/pJAQ=       1\n",
      "jZpgaJGkUb/CMa8kQX5bdgLJGgCyejxINoUNiiG+P8o=       1\n",
      "M6rqdn4hhc+IZyHuRfBcf/zcxrq1koy/gY/c7hgz2so=       1\n",
      "Name: msno, Length: 25131, dtype: int64\n",
      "\n",
      "song_id属性有224753的不同取值，各取值及其出现的次数\n",
      "\n",
      "ZcKgNis1AP1LA0sdtIddrtk7P04iiJzJrXvwXdT/X3Q=    8320\n",
      "DLewxl82pFT6+OgSpGYmN26IayVLtcv2eiJEJBSIWLI=    5876\n",
      "CsqfeIr3/stOaLTq+Z+Yt6VnPSqPWS2wxxnCKJL86oM=    5585\n",
      "ZXW6Df8q6jp6cUWCM6hFTAG/3PrqUZ2j5cBobvszX6E=    5541\n",
      "HGROX/ydkner7ykeMTZjh5oW/b0DbHb0awQIqRNBHIc=    5175\n",
      "CIczx2LF40xrm4tpnNvN7YbzZ4+pzTimo00/fjj+4CM=    4903\n",
      "fMNUtrU1ABmSjdJy+XQlo0QP/4Sf+XEAWZkQ5mZwSm0=    4649\n",
      "ld7wZZNHXEO8wGLYyGti0q6ibv/knCVDTyhJImWn5Mc=    4108\n",
      "UQeOwfhcqgEcIwp3cgNiLGW1237Qjpvqzt/asQimVp0=    3688\n",
      "YDH6i3TaG+cGzSoSx7oIWZrmxPs8ANyO5shZj+j9obg=    3638\n",
      "C5eT8KL1Fd76QFTuivkeB9xFdGo60dI80JIr0IP3Foo=    3575\n",
      "yk2hZS4SA2yit4orR06WHYDkkXg5+E77bA7Y533znZ4=    3265\n",
      "kMIJnH+Xma633KX+coW+4Jy517FE7ayQfvEWue2knfQ=    3143\n",
      "4rJZSt5ePuVVxbTxaDMqYNMGlH4HIQCQ+Ypc/uKfp6I=    3125\n",
      "Kcpjorb5iiPF0KBDW9KwghGP6pOa0kmflHJ9QPW8yh0=    3110\n",
      "JgBVDRyiREdaf2XOliIsY0CCMniIuBCtfMA6WzrWnck=    3099\n",
      "+2Ds+tDr0QXQ5vtW+ygssPP/Fm9YFp6iELmpex7pLcU=    3059\n",
      "i7mW3vGbMipgf/HXBI3oNWdE5yRdRnytiWCRnplEDTE=    3057\n",
      "KCJ8BlSfRQRgB7EVuzFvg52AhR8m2fT032MzN5ewbEI=    2973\n",
      "2uFTn3kDES2C3JeqzV4cpSAGQ756GFK0XajzEDLqER4=    2972\n",
      "0CBHr70SNEHsVP/3sEVonkupy3NNCg2H0f5SxKEM4Rc=    2955\n",
      "QiRTMce4Umbhcl9MneQyM5liKG9ZLNOpQ0OK4jfhwyY=    2912\n",
      "0b8M2FLEH176y/ng1rK9ftUYDYnA+1NdoCOUeuyBMtU=    2906\n",
      "JVwoBKjxxE9L/IpNKsSCJJIYHcxvwxOKbFnM3TLwcTQ=    2869\n",
      "Ujt3R5hAe0Pix2mURuC+CO6aazhCvEU2lByPzy3E90k=    2837\n",
      "KnNPuXCvhpBA4L6baLLiQnVi6RMEN0EoHN6gwqMkxh8=    2756\n",
      "G0rUz0kguI35yzj95LX2BXI30PqCcJJtPNfirR0BhIs=    2720\n",
      "UVARDmPZQIC2uety0QFm1vC5JvkqsezXsSnfc3r1WXc=    2648\n",
      "wJoWzZd7AL+qX9xZWZwRTzGRYg0Lxcl5Pe+9n5hZgAQ=    2647\n",
      "LUD48hW5OjRtlnREnobe+RbZ8y4o+EDkr1Uxlv7CBAU=    2585\n",
      "                                                ... \n",
      "6hTZ2IOoLS7MEBwVWzqkBzIhZaWpop67Sei/Q2YApBc=       1\n",
      "6e32TCysQsbYeuX+zVxy6wlXTpt2b8K8QW2zbza+6Sk=       1\n",
      "6cxCYEAuTg3Vmsu862rWcvw6NfWHilaYe0lWqdqC6r0=       1\n",
      "6b11rZSHyoqCHjV4H3FRowmC9si+HoraU+FoNHY9jSY=       1\n",
      "6avcUatj5WV/BoQaAT0jeZ2Wg65fBtU8LnNVwHJ+zS4=       1\n",
      "6a0NqV0gxjIaJ1l9aYsGFU/8QPR/h+e8o87ssQcB2rQ=       1\n",
      "6ZkO0xZkQAXaUN2mIpDUmmSJ0E658u50WoQaJ02guhU=       1\n",
      "6oyFJdPiKq+3ydJcqj/fF0QJ4dKb70/kX+qrmKUrRtw=       1\n",
      "6pmGfHUWaiN0s004fs6PC27OrCrnyhMY05C0+PVvKb4=       1\n",
      "7+hs3H4XQa2XRACFy1RZNGyD1KGqJRkYFdEiNI3KsxU=       1\n",
      "6vLYA4byPm3kbXsv0mQf9JX2unSEeBtdooc6STgFmfE=       1\n",
      "7+1xSL6xAHNXelBxM9eNV5qobSuutFec9nIe99AdGPk=       1\n",
      "6zDiTKI36vstaVhz/X4l5JAZm0YY+jwNColokUV4zUQ=       1\n",
      "6z5lR9B/Zm6d0+alQUdzS5ZosLqQOxjO7d8frpY1OXI=       1\n",
      "6yBtS7NwTXYd9IBM8UoQUKEsmnYU7FwtMzNppPe1nzc=       1\n",
      "6y+QnamqAM2wKdbx/AZKxGt2V2uPt3BijWaT4nBD3OA=       1\n",
      "6x0skKfqXdwd5G8Xdlhzg/Vzi/GOkQiLcMf7et/0TGY=       1\n",
      "6vfsmyNCpxnV7fUnpf26Dp1CH82On1Nd5EYNSE1b0K0=       1\n",
      "6vZufFZS8ZK4xQG0XYQ92DjIoj8i6aA2691Yyr70480=       1\n",
      "6v62eiMTbj/nQhTD7izcUMSozg/5Pw51PJWmettXxKs=       1\n",
      "6qx1HI8UwmQ9wUeE6hcSYKzx0aIO46wQsdHrgpANvRs=       1\n",
      "6umvQXOgPNfeqs+nUz4DSog+Dib8b6CaNzPWmO7dJFY=       1\n",
      "6ujQggB9zN5TLBY8iHBpymhGmTPR5sGCAXo5NTGgyPw=       1\n",
      "6udbGujfWXcotBOpJ87vkvC5keexQDHA5fDy9X0yhAQ=       1\n",
      "6u8iWg8f6TSnHlUCCMXde4xzzt1sn+bC7ahYl3Jr6I8=       1\n",
      "6tvw2xxPptqTHze6COlND9mrzbt7QdQNiGeIOULor8I=       1\n",
      "6sury+NtUmr2FJtZBJ4fssFB3BtVH3Q42Pmlkfx9OiY=       1\n",
      "6sB6mNM8DciDdMUWOZ0CL8sSKinXPt42aYxHSZ8hgF8=       1\n",
      "6r8PxL3BgTZ0t384cp2z3SFY8uHDK3fTOdXCgWABkBY=       1\n",
      "dh+TIXgsY8yFO7/GfcGwFu5PoJC2oD2OGRF2JQY0nFQ=       1\n",
      "Name: song_id, Length: 224753, dtype: int64\n",
      "\n",
      "source_system_tab属性有9的不同取值，各取值及其出现的次数\n",
      "\n",
      "my library      1019492\n",
      "discover         871068\n",
      "search           277615\n",
      "radio            212765\n",
      "listen with       98628\n",
      "explore           66023\n",
      "notification       2124\n",
      "settings            633\n",
      "Name: source_system_tab, dtype: int64\n",
      "\n",
      "source_screen_name属性有23的不同取值，各取值及其出现的次数\n",
      "\n",
      "Local playlist more     845115\n",
      "Online playlist more    529807\n",
      "Radio                   211201\n",
      "Album more              176129\n",
      "Search                  121982\n",
      "Artist more             110999\n",
      "Discover Feature         93401\n",
      "Others profile more      90457\n",
      "Discover Chart           78999\n",
      "Discover Genre           41617\n",
      "Explore                  27872\n",
      "My library               25559\n",
      "Unknown                  23620\n",
      "Discover New              5277\n",
      "Search Trends             4883\n",
      "Search Home               4705\n",
      "My library_Search         2114\n",
      "Self profile more          131\n",
      "People local                13\n",
      "Concert                     13\n",
      "Payment                     12\n",
      "People global                1\n",
      "Name: source_screen_name, dtype: int64\n",
      "\n",
      "source_type属性有13的不同取值，各取值及其出现的次数\n",
      "\n",
      "online-playlist           774532\n",
      "local-library             582346\n",
      "local-playlist            294537\n",
      "radio                     215164\n",
      "album                     195190\n",
      "top-hits-for-artist       179360\n",
      "song                      129153\n",
      "song-based-playlist        87179\n",
      "listen-with                84499\n",
      "topic-article-playlist      5082\n",
      "my-daily-playlist           2023\n",
      "artist                       428\n",
      "Name: source_type, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "for col in list(test.columns):\n",
    "    num_vlaules = len(test[col].unique())\n",
    "    print('\\n%s属性有%d的不同取值，各取值及其出现的次数\\n'% (col,num_vlaules))\n",
    "    print(test[col].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "source_screen_name测试数据比训练数据多出两类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "msno属性有34403的不同取值，各取值及其出现的次数\n",
      "\n",
      "nX1j+J2hTLU9jaDItgMMkdAjGOREjme1s22FuMzCEAw=    1\n",
      "wsQMD7DV58v23iUelyQZ4p9Lsk5Rwe5hijZZIjXBuU4=    1\n",
      "Z0zTir0RluA0cPqJpvHA8GmzNhCXuthQGBkxc6l803M=    1\n",
      "5yil2dxbsRX8P4SwKHYgp3H1+0X1teiDPxVaDGkBzRU=    1\n",
      "NdeQueZTUXZnn6kTINQHmLeGRhDmR9K1N5sjxyX5SMA=    1\n",
      "7i3YfUiCrEOynAhbKVudqYwvUqfy2F/ut6fVGQWhij8=    1\n",
      "OhvGKDNSOSL4+L676lk2y0pUrhHfz5YmSQfWFg69zVk=    1\n",
      "bnvybZgwaC0kASLxdAvaukX7D5HR/SgIK2TUVoIoNhI=    1\n",
      "sZbcFNPmOCoJAYECN+27u6XDiIeLrwPMBOLzF4nWkTc=    1\n",
      "FVc3GVCKmwQvlymhSbbj5XnQD2Ngteu/fgvgp6pY1U0=    1\n",
      "Hc68lZkCdHil4MGh4KTLQr31jFQjKgBHxqpA0mCtgd8=    1\n",
      "9gWqcd1n2iEvYTOzvUV7kMT2diPvXE6e6CowhpMzIZo=    1\n",
      "LigKORoHn7g04jWYrRErZqrY/fRp4zCiFJvBArOnb6M=    1\n",
      "jbLnZiGfyBLtlQRNBWGa+HTx27Bd3k6QHMt0Dttda1Q=    1\n",
      "LgWffgUropU8x5ulxSTtCi8ajb2NrvM2VMx4793RYzk=    1\n",
      "GWfPAqH3Y/81FT4ORJyR9v0ulwJ8w01/oM71uHjJ10I=    1\n",
      "IsaA+NoWimcR6eEAR2zIGKq+1+Z96WUJTsc3/CGZ30c=    1\n",
      "VUw+xnkMDLMoLEclUQJbykWaSvg795wksSqfoKqa9ME=    1\n",
      "bB6YXjj1AamYV6l/RWYgprqPPc9pYPiq28tImmz/kMQ=    1\n",
      "/ARRvpNR4P9BF+qmzimvwbmmVZMxRBxzB0alQ/v7y2o=    1\n",
      "IwOvAt8EU0AvtmlCIw97f0YZ3zqUHtuVBEo7dbg4Dgc=    1\n",
      "Vcjy/gJ4ed5/+KuG/Lrz7AwC9lNLAwWhsz2hcBb0sac=    1\n",
      "Ly+hUu353rti8MJ/V8UPgOruHdon7AXRq37RufUFNf4=    1\n",
      "QqHFdWP9aDuqJaMfaNYHrAhVVv+syGkh/ZSf4h/MgOg=    1\n",
      "PXXrSmwqp32WtDevNaMsN8YDX71Bklz6seDlFdbFie0=    1\n",
      "yLPstMcz7Q3oIta4aDpNVDMFyrXSvPjoMdhMLgV9jZU=    1\n",
      "Nk3z0HQk3wuNZJ5DtDAmxoiF3qGKL5knFAWzrRhg/MA=    1\n",
      "jXNCDt3onbfWpY58c2F9+LZgn5rnX3H0nk22rduEiZ8=    1\n",
      "4BM79j6o9P5MDaSEL7Uh7FNDgmYMsuax1yLsN15/vTE=    1\n",
      "nPssk4Yik/A0HQxjbmiQ0o3ZBBH6F11aRCRe2hZgAgc=    1\n",
      "                                               ..\n",
      "DgyuqbzpZQ5SAt/54ZQhHrsiBFCo7PzWQafLsc5vQGM=    1\n",
      "qQEDz/Sux7HwmT1GfHi75kc/ERTPg3P15Gm0+lAzRKw=    1\n",
      "RP6IfimmUEXbNvu1F1TcIu916wmuR5PyZcsXIkyQ46M=    1\n",
      "fRp1bUMKFZVmPava7t7GnHssgFxZ5L2eboxEuULimaA=    1\n",
      "/kFdEFK+4mhAoYDkkoZqBr2MOYIomNtVxYGSYZPdEJg=    1\n",
      "YImNJINr+xHl8ME1GF8NJU1EEj3yZrbjdwbxpQqlCEg=    1\n",
      "qzBE+D+8tZjIJs/vexZQbHPxFF3usJBaYX7/K+XhrpU=    1\n",
      "NZMIxvlYjg/FNWykxNIxKLkyfRVuKW283pW8hpj2pho=    1\n",
      "LgTNkY94MqdnP1x61KtqeWLZt8pEgZOpeAvQaZgKXVc=    1\n",
      "LX/HOc7qQPED5LFsO/HV36nb6fIUumE5OvgYL8e+hRM=    1\n",
      "5GokfrtZUvkqYIXHAjjSw611Dlel8it+NWjW/pgo470=    1\n",
      "yC7Y1/fjrPxli+boTsLkmPNaOpLzM6davSupV1c90uw=    1\n",
      "oxBNCENj9XjX26hOiwSOCTIUPp3WqlvbDkQTIB6szpE=    1\n",
      "e+yxUBslKjR8Yvlx7Bc+MW5dooFr3ycHhkO/5xP9XQE=    1\n",
      "uUexAmKz1HzizQxKyi3z0DT2gW6R2T0pPZAnnraKdOQ=    1\n",
      "nzE2b2mZsrb+NmvA6jvuPkYkR2M2O3SBTFhxmGG1j0k=    1\n",
      "0z9NrBr1XnsvyoTUvR6VVwCsUJsM7iVRkBy0vUGPO0w=    1\n",
      "jKXrrXm1znyQm04WiDVPu+C0cuEIeAkv6rl9agGsRYk=    1\n",
      "7YOF8/e5yw5HEVz28Av01WXBNO9jpw13GN1uNIKNy9A=    1\n",
      "rgs7ofqrZf/YcqRxB3R4lzhfWBE7jKbq/Vs1jlmThUk=    1\n",
      "ck5e8ug38PfVT3zvGYLRMKnchq4pScEw7g8/d4OThBo=    1\n",
      "nI5EI2M7KX0/FCDqe55STA9yq339axN3OHRyHz52r6U=    1\n",
      "Ek1HX8G00M1TFMFTsm8W56ik4Qn3JTfho2hmxjoDihU=    1\n",
      "1vtM4P0ikaGFHWHautVny/r0Bno6JpQufS5fUXXzuCo=    1\n",
      "0a1lABrEJWU3mNFOYTmlHBagkqejty9kZY4uP3Z/Muk=    1\n",
      "ajV22lcXecBsJOb9LFBQhW+QLQn9SuTiGXqdqI2zwvY=    1\n",
      "ZgHmpR2RZiRuWDfKWT+fUYjyDTOFMMrE5j3h1+ihM30=    1\n",
      "YqfGwgsZ8SIqxehgGTJzhudZhIO9IrvDiJOtNH19cIM=    1\n",
      "+mN1nGXWFY04wCApy3Gp3CAB84Nrki+i6ggs8OmTXBw=    1\n",
      "LJraZezHPRvKj9OLzXA8QGDaTz2PuPflMnKbLD52Sng=    1\n",
      "Name: msno, Length: 34403, dtype: int64\n",
      "\n",
      "city属性有21的不同取值，各取值及其出现的次数\n",
      "\n",
      "1     19445\n",
      "13     3395\n",
      "5      2634\n",
      "4      1732\n",
      "15     1525\n",
      "22     1467\n",
      "6       913\n",
      "14      708\n",
      "12      491\n",
      "9       309\n",
      "8       289\n",
      "11      285\n",
      "18      259\n",
      "10      216\n",
      "21      213\n",
      "3       204\n",
      "17      152\n",
      "7        93\n",
      "16       35\n",
      "20       27\n",
      "19       11\n",
      "Name: city, dtype: int64\n",
      "\n",
      "bd属性有94的不同取值，各取值及其出现的次数\n",
      "\n",
      "0      19932\n",
      "27       751\n",
      "22       751\n",
      "24       740\n",
      "26       719\n",
      "25       716\n",
      "23       712\n",
      "28       688\n",
      "21       685\n",
      "29       661\n",
      "20       631\n",
      "30       602\n",
      "19       507\n",
      "31       491\n",
      "32       466\n",
      "18       466\n",
      "33       416\n",
      "34       404\n",
      "17       398\n",
      "35       380\n",
      "36       341\n",
      "37       300\n",
      "38       294\n",
      "39       226\n",
      "16       215\n",
      "40       204\n",
      "41       194\n",
      "44       138\n",
      "42       131\n",
      "43       121\n",
      "       ...  \n",
      "5          2\n",
      "102        2\n",
      "96         1\n",
      "97         1\n",
      "2          1\n",
      "163        1\n",
      "131        1\n",
      "3          1\n",
      "101        1\n",
      "95         1\n",
      "6          1\n",
      "70         1\n",
      "93         1\n",
      "218        1\n",
      "90         1\n",
      "89         1\n",
      "87         1\n",
      "213        1\n",
      "85         1\n",
      "82         1\n",
      "144        1\n",
      "76         1\n",
      "12         1\n",
      "107        1\n",
      "11         1\n",
      "106        1\n",
      "10         1\n",
      "103        1\n",
      "7          1\n",
      "78         1\n",
      "Name: bd, Length: 94, dtype: int64\n",
      "\n",
      "gender属性有3的不同取值，各取值及其出现的次数\n",
      "\n",
      "male      7405\n",
      "female    7096\n",
      "Name: gender, dtype: int64\n",
      "\n",
      "registered_via属性有6的不同取值，各取值及其出现的次数\n",
      "\n",
      "4     11392\n",
      "7      9433\n",
      "9      8628\n",
      "3      4879\n",
      "13       70\n",
      "16        1\n",
      "Name: registered_via, dtype: int64\n",
      "\n",
      "registration_init_time属性有3862的不同取值，各取值及其出现的次数\n",
      "\n",
      "2017-01-26    117\n",
      "2016-12-03    116\n",
      "2016-12-31    113\n",
      "2017-01-22    111\n",
      "2017-01-21    109\n",
      "2017-01-02    107\n",
      "2016-12-04    106\n",
      "2016-12-17    106\n",
      "2017-01-20    104\n",
      "2016-12-24    104\n",
      "2016-12-11    103\n",
      "2017-02-04    103\n",
      "2016-12-18    102\n",
      "2017-01-27    100\n",
      "2017-01-08     99\n",
      "2017-01-31     98\n",
      "2017-01-07     98\n",
      "2016-12-25     98\n",
      "2016-12-02     97\n",
      "2016-12-10     95\n",
      "2017-02-02     94\n",
      "2017-01-30     94\n",
      "2016-12-12     93\n",
      "2016-12-27     93\n",
      "2017-02-27     93\n",
      "2017-02-03     92\n",
      "2017-01-14     92\n",
      "2016-12-30     91\n",
      "2017-02-25     91\n",
      "2017-02-07     90\n",
      "             ... \n",
      "2005-07-06      1\n",
      "2009-03-26      1\n",
      "2009-02-14      1\n",
      "2006-07-23      1\n",
      "2005-03-04      1\n",
      "2008-12-19      1\n",
      "2004-09-09      1\n",
      "2008-01-08      1\n",
      "2009-03-16      1\n",
      "2005-08-08      1\n",
      "2005-11-10      1\n",
      "2009-12-23      1\n",
      "2010-02-14      1\n",
      "2009-08-20      1\n",
      "2005-03-17      1\n",
      "2008-06-27      1\n",
      "2005-03-08      1\n",
      "2009-09-01      1\n",
      "2008-02-13      1\n",
      "2008-08-02      1\n",
      "2009-07-01      1\n",
      "2008-12-23      1\n",
      "2009-07-06      1\n",
      "2009-03-30      1\n",
      "2007-02-09      1\n",
      "2006-08-03      1\n",
      "2009-10-14      1\n",
      "2010-07-03      1\n",
      "2007-06-01      1\n",
      "2004-04-02      1\n",
      "Name: registration_init_time, Length: 3862, dtype: int64\n",
      "\n",
      "expiration_date属性有1484的不同取值，各取值及其出现的次数\n",
      "\n",
      "2017-09-30    2498\n",
      "2017-10-04     551\n",
      "2017-10-05     549\n",
      "2017-10-06     515\n",
      "2017-10-01     495\n",
      "2017-10-02     494\n",
      "2017-09-17     481\n",
      "2017-09-16     480\n",
      "2017-10-03     460\n",
      "2017-09-10     447\n",
      "2017-09-20     447\n",
      "2017-09-19     433\n",
      "2017-09-09     427\n",
      "2017-09-12     416\n",
      "2017-09-22     412\n",
      "2017-09-21     412\n",
      "2017-09-13     412\n",
      "2017-09-18     406\n",
      "2017-09-23     399\n",
      "2017-09-08     396\n",
      "2017-09-11     392\n",
      "2017-09-24     388\n",
      "2017-10-07     386\n",
      "2017-09-15     378\n",
      "2017-09-14     372\n",
      "2017-07-08     255\n",
      "2017-09-25     235\n",
      "2017-06-09     214\n",
      "2017-06-10     196\n",
      "2017-09-26     195\n",
      "              ... \n",
      "2015-04-21       1\n",
      "2015-04-26       1\n",
      "2007-05-14       1\n",
      "2012-07-02       1\n",
      "2013-02-02       1\n",
      "2009-02-10       1\n",
      "2019-05-28       1\n",
      "2015-05-30       1\n",
      "2015-12-31       1\n",
      "2015-06-14       1\n",
      "2016-04-03       1\n",
      "2014-06-07       1\n",
      "2014-04-29       1\n",
      "2015-05-17       1\n",
      "2018-10-26       1\n",
      "2008-07-17       1\n",
      "2012-07-22       1\n",
      "2014-04-15       1\n",
      "2012-01-02       1\n",
      "2016-01-07       1\n",
      "2011-06-14       1\n",
      "2015-07-28       1\n",
      "2014-10-03       1\n",
      "2013-08-18       1\n",
      "2014-12-25       1\n",
      "2014-06-06       1\n",
      "2013-11-16       1\n",
      "2019-08-28       1\n",
      "2014-07-15       1\n",
      "1970-01-01       1\n",
      "Name: expiration_date, Length: 1484, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "for col in list(members.columns):\n",
    "    num_vlaules = len(members[col].unique())\n",
    "    print('\\n%s属性有%d的不同取值，各取值及其出现的次数\\n'% (col,num_vlaules))\n",
    "    print(members[col].value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "song_id属性有2296320的不同取值，各取值及其出现的次数\n",
      "\n",
      "zzveIwHI6xUE0OUV/bqxS5B3QnPCoBkNGCp7xLdTt4U=    1\n",
      "pmh888U51EQ3hQYeVPhal9UafIQOk8v2ZBrptEOwmz0=    1\n",
      "pmo73+CzvHKvJUeLIDII2jhV5Ol6VfWJLoGLSsrcgrE=    1\n",
      "pmmxJUJiQrTlQ+ybH45+mQVMjaoDLNDbUysVzQBg4MQ=    1\n",
      "pmlrHN1x1fyoHBh2rXZtXwZr34ZlJb0qa8ceHBiLbbw=    1\n",
      "pmkOB/GVXbIMpbMQCsT+3hLtkAG+Ut2n57KNHJIWSGY=    1\n",
      "pmhjpDFI3bGF0sIj8XSl86bexc3CcBIBs0cQ75DKT4o=    1\n",
      "pmh8ceUiEM0chtksd4BN+4Xj7Ll7/cuZcsaiwKDwMII=    1\n",
      "pmZAQUHSnKVzR1VtZUuz5Zi8CReOK7ro15bmbqx3w7I=    1\n",
      "pmp1m5MS9zKIldQYhedu9i7QJvJyFTtG+yHyggf7e4M=    1\n",
      "pmXmCFdHV3LfFccx/7ZjWvUI4uP3Ow6IhSBHbPXJ9Dg=    1\n",
      "pmSdmoFu3d3rI4TXNG0FOzkX/Tl7xuETITE5qyYQmqw=    1\n",
      "pmMXYXOIwNXZBxYrEzoTNt5e9joUBMHoJBgh6yILgIQ=    1\n",
      "pmKXSC8z0NJRIMqfFYcV2SM5j0eiDBaFZmnY+wRjxYU=    1\n",
      "pmK1TAR3i9iEwTnEqhhszoFIjj/83keDs9cJL+5CJhE=    1\n",
      "pmJn0yDSpflwx1hKrbUfXJig5harQ2y915IZdajW25k=    1\n",
      "pmorj2iVPrR8DoA9z8SZ9Ud2a6g6D/349N3Nx2tP9z4=    1\n",
      "pmva85DAFr5VzdVBq+6Z3yb7e5vWmwHRdLxNg21ApXo=    1\n",
      "pgEcekZUIWayZ8Cr8aZrC5INo0f2kEnyju5R4arHUzs=    1\n",
      "pnAyJs8DbqVXdSXz1BmrZBGTvXCOjYdQ+xfqregvjgg=    1\n",
      "pnGyoyGFX41xem1HI9r5P++zXPWA4IoZCQxruPLf4T0=    1\n",
      "pnFmJPWrOIUeblst1djO+9srjLyEKgd2RgZgoRmPU7g=    1\n",
      "pnFWLQaMVhO/MsGbZMKKvSRejiTRdzfxB8dyojXaEnk=    1\n",
      "pnDtAbCJ9Vva7c3mwMwIrQl94uN+wtm4tV6JvRqaPME=    1\n",
      "pnBVKv4Vi63hrUzt7uDkwcAmq3zNjf6MPAkL7+KS+So=    1\n",
      "pnB1iX3909muPVPljD8Hbl6lt6yey7GDnVYxGzIavEo=    1\n",
      "pn957+BCLdW2M037Xpb1+D+GJ0R+MHMUqkDXnZDTCuI=    1\n",
      "pmvfVt4oujvRk/XCFp4JofducG9ijn/Ys0gm3RdNr6o=    1\n",
      "pn8qqi7JXBArzfVofWleZ+pa1IQJh/KE+u5cDoI8nT0=    1\n",
      "pn8PjmSzQCoqdG6CEK1Rg2PDfq5aTHFZgj+b84j/oNs=    1\n",
      "                                               ..\n",
      "fYFFjVVOWqSKxqeRMtXtzYrM75YiNY12iD3u+krDOiM=    1\n",
      "fYF7oHI+CjOc7PpnIMlrbWt5nCLS9RBzinX/MmOCV44=    1\n",
      "fYD99AhLbq1TzdmqPM07sTaZ/lalq9Fgt+yRRjuO8Y0=    1\n",
      "fYBzBpACnrLS9p6V7zMxO8XR+7wrD5SwoSUSMwMlW+I=    1\n",
      "fY8rG2VbAS2j/MxO6LxTrvbUPy8uu1ktFWGiLg6VSZI=    1\n",
      "fYmHnLjYGP/juHWEZylvegoMqRpxJBlK7KgqmS1nozE=    1\n",
      "fY7d01zmxDytVNi9T4OOkCdR4h7HnaqKqGEuGHS706k=    1\n",
      "fY5Z+K9MHPM7w7mZO/pb/UXSPxMU3K89ltUP08kmB28=    1\n",
      "fY1c0xoK94xoqQqNZjmLYqpcQvQ/CPs37h4zT9w2UY4=    1\n",
      "fY+Rc3OEFDd8Yg0QACyG+mXBNaL60eM3gmgDjxNOggo=    1\n",
      "fY+PSQoFuvEEL8vblFxNouwNRMnq0MWOVyYArHJjhOA=    1\n",
      "fXy0LcV7yYGm693SetTu2EZwzhyaYk0f14EyNTEV4mE=    1\n",
      "fYKYFNjcrPTQgvIdqwuQ7u4667eMGx46eBRECeyBvz8=    1\n",
      "fYLsZejs5Ztce3DJJrA8B2hnkgVgG2PKUgZ1fQnon0g=    1\n",
      "fYMHiBUycArXc4hHAnjslhQlg0usCjdiPpuyBa0TX+g=    1\n",
      "fYQzTbLWBfAlytn8468wKDImd+voFyEX9iHrVD69jMY=    1\n",
      "fYU+u6cqylhmaDz8xCvuHbNNRkJBzgzDFuty6/LO9A8=    1\n",
      "fYVs0lQoIimBSnqGLl/ZZG9injXZee+HMFuxpFXN9Ao=    1\n",
      "fYVucyR9Lto2JS0gMbKAo6BVdk4QREzhnhAJwRsPlw0=    1\n",
      "fYW17LHysTeQe3qZq8yXOyNr3up3mn/C0ItlfJ2pzH0=    1\n",
      "fYW6AGTVORvVvEFnjyRImokyWOih5OBFNHj7QccT/HY=    1\n",
      "fYWTo57QLtCfrqQvURWomx0bHjpiOzMchaGCt0L6c8Y=    1\n",
      "fYWYDchPi8+GUNdeosoW1+wB4FE1goNS6BzLMduc0gE=    1\n",
      "fYYFeMtcbk2DDhPH2e7Mdo4doZesnS7naeJ1+FOWun8=    1\n",
      "fYbyOhdcl9luft1+RkhdcsPOtXxHRnX8rdEb6DAB9p4=    1\n",
      "fYehr+ZJzY+P8PBLIDVX2CwCMm+bIyx759HBH3ApiTQ=    1\n",
      "fYfS3qb7QLngWUhPwLpz5ujaSRbYwiE7xFfUu+9vv/g=    1\n",
      "fYhy2tfdVc9Mtr38drDaRxSw0eASLIsG0WorohsuSWo=    1\n",
      "fYj+s9tiJxZiGet3iwuALHBXoUnN8bmKZyb9kY95oUw=    1\n",
      "++/ACCkEN/+VtgrJxEqeRgRmV4y8pcarDJ9T/yRAi1E=    1\n",
      "Name: song_id, Length: 2296320, dtype: int64\n",
      "\n",
      "song_length属性有146534的不同取值，各取值及其出现的次数\n",
      "\n",
      "239908     1763\n",
      "203964     1419\n",
      "213995     1393\n",
      "254955     1103\n",
      "233221     1053\n",
      "193933     1024\n",
      "224026     1023\n",
      "223190      985\n",
      "244924      965\n",
      "222354      946\n",
      "221518      945\n",
      "224862      936\n",
      "214831      935\n",
      "208979      932\n",
      "211487      915\n",
      "198948      912\n",
      "234057      903\n",
      "189753      899\n",
      "209908      894\n",
      "212323      888\n",
      "204800      887\n",
      "234893      887\n",
      "244088      883\n",
      "201456      883\n",
      "206471      882\n",
      "213159      882\n",
      "240744      881\n",
      "199784      877\n",
      "193097      869\n",
      "183066      864\n",
      "           ... \n",
      "392658        1\n",
      "492957        1\n",
      "274945        1\n",
      "425378        1\n",
      "411045        1\n",
      "1114279       1\n",
      "1314821       1\n",
      "439739        1\n",
      "728037        1\n",
      "689110        1\n",
      "682955        1\n",
      "644028        1\n",
      "333263        1\n",
      "623542        1\n",
      "318966        1\n",
      "226024        1\n",
      "553864        1\n",
      "1024882       1\n",
      "979800        1\n",
      "296445        1\n",
      "748547        1\n",
      "732171        1\n",
      "762906        1\n",
      "232183        1\n",
      "756767        1\n",
      "238322        1\n",
      "244465        1\n",
      "219885        1\n",
      "926890        1\n",
      "8196          1\n",
      "Name: song_length, Length: 146534, dtype: int64\n",
      "\n",
      "genre_ids属性有1046的不同取值，各取值及其出现的次数\n",
      "\n",
      "465                     567911\n",
      "958                     176349\n",
      "2022                    168870\n",
      "1609                    166457\n",
      "2122                    139938\n",
      "1259                    101485\n",
      "921                      67578\n",
      "1152                     48730\n",
      "359                      43601\n",
      "786                      42999\n",
      "726                      34969\n",
      "139                      34876\n",
      "1011                     34013\n",
      "940                      33490\n",
      "1572|275                 24338\n",
      "1955                     20980\n",
      "691                      19472\n",
      "139|125|109              17613\n",
      "873                      17542\n",
      "437                      17212\n",
      "947                      17106\n",
      "388                      16776\n",
      "458                      15438\n",
      "444                      14859\n",
      "1616                     14188\n",
      "242                      13756\n",
      "451                      13283\n",
      "880                      13059\n",
      "423                      11814\n",
      "829                      11759\n",
      "                         ...  \n",
      "465|1259|139                 1\n",
      "880|139                      1\n",
      "880|430                      1\n",
      "958|1011                     1\n",
      "921|829                      1\n",
      "921|2189|798                 1\n",
      "465|1981                     1\n",
      "458|2130                     1\n",
      "465|2022|139|125|109         1\n",
      "921|1955                     1\n",
      "921|545                      1\n",
      "1609|2079                    1\n",
      "1969|1011|2100               1\n",
      "242|359                      1\n",
      "430|1259                     1\n",
      "458|451                      1\n",
      "458|698                      1\n",
      "545|873                      1\n",
      "829|921                      1\n",
      "786|2122|2086|374            1\n",
      "1152|205                     1\n",
      "2065|2191                    1\n",
      "2189|958|798                 1\n",
      "388|430                      1\n",
      "423|691                      1\n",
      "465|109|958                  1\n",
      "465|1138|829                 1\n",
      "465|275                      1\n",
      "798|1011                     1\n",
      "873|1609                     1\n",
      "Name: genre_ids, Length: 1045, dtype: int64\n",
      "\n",
      "artist_name属性有222363的不同取值，各取值及其出现的次数\n",
      "\n",
      "Various Artists                                   145916\n",
      "証聲音樂圖書館 ECHO MUSIC                                 11276\n",
      "Billy Vaughn                                        4828\n",
      "รวมศิลปิน                                           4432\n",
      "Richard Clayderman                                  4180\n",
      "Elvis Presley                                       4038\n",
      "Nat King Cole                                       3806\n",
      "Billie Holiday                                      3674\n",
      "Frank Sinatra                                       3594\n",
      "Armin van Buuren                                    2818\n",
      "Various                                             2742\n",
      "Ella Fitzgerald                                     2721\n",
      "オルゴールサウンド J-POP                                     2692\n",
      "Glenn Gould                                         2689\n",
      "Bill Evans                                          2572\n",
      "Chet Baker                                          2532\n",
      "鄧麗君 (Teresa Teng)                                   2487\n",
      "Bob Dylan                                           2471\n",
      "貴族精選                                                2419\n",
      "Ray Charles                                         2235\n",
      "The Kiboomers                                       2230\n",
      "Michael Jackson                                     2208\n",
      "Miles Davis                                         2179\n",
      "Falcom Sound Team jdk                               2164\n",
      "Louis Armstrong                                     2161\n",
      "Wolfgang Amadeus Mozart                             2047\n",
      "Andy Williams                                       1995\n",
      "Power Music Workout                                 1987\n",
      "霹靂布袋戲劇集原聲帶                                          1905\n",
      "Julie London                                        1838\n",
      "                                                   ...  \n",
      "Olimpia Rey de Copas                                   1\n",
      "Oligarshiiit                                           1\n",
      "Olga Komarova                                          1\n",
      "Olga Jegunova                                          1\n",
      "Oleg Katsura & Band Klass                              1\n",
      "Oleg Kagan| Natalia Gutman| Sviatoslav Richter         1\n",
      "Old Time Relijun                                       1\n",
      "Olavo Bilac                                            1\n",
      "Omni Alien| Laroo T.H.H.                               1\n",
      "Omnia| Audrey Gallagher                                1\n",
      "Omniverse                                              1\n",
      "On Your Horizon                                        1\n",
      "Orange Cloud                                           1\n",
      "Oran                                                   1\n",
      "Opossum                                                1\n",
      "Ophelia Dore                                           1\n",
      "Opal Puckett                                           1\n",
      "Oola da Boss                                           1\n",
      "Onyx Brass                                             1\n",
      "Onyenze                                                1\n",
      "Only C                                                 1\n",
      "OneHundred Proof                                       1\n",
      "One Short Fall                                         1\n",
      "One Love Devotional Chant                              1\n",
      "One For The Team                                       1\n",
      "One Culture                                            1\n",
      "One Bit                                                1\n",
      "One & Only                                             1\n",
      "Oncl' Syl'                                             1\n",
      "Vilou goes Swing                                       1\n",
      "Name: artist_name, Length: 222363, dtype: int64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "composer属性有329824的不同取值，各取值及其出现的次数\n",
      "\n",
      "Neuromancer                                                                                                                                                                                             17888\n",
      "Johann Sebastian Bach                                                                                                                                                                                   12105\n",
      "Wolfgang Amadeus Mozart                                                                                                                                                                                 10839\n",
      "Marco Rinaldo                                                                                                                                                                                           10803\n",
      "Traditional                                                                                                                                                                                             10105\n",
      "Ludwig van Beethoven                                                                                                                                                                                     7290\n",
      "佚名                                                                                                                                                                                                       6259\n",
      "Sacchetto G.                                                                                                                                                                                             5479\n",
      "Frédéric Chopin                                                                                                                                                                                          5468\n",
      "Antonio Vivaldi                                                                                                                                                                                          3376\n",
      "Unknown                                                                                                                                                                                                  3334\n",
      "Robert Schumann                                                                                                                                                                                          3138\n",
      "DR                                                                                                                                                                                                       3107\n",
      "Johannes Brahms                                                                                                                                                                                          2903\n",
      "Franz Schubert                                                                                                                                                                                           2729\n",
      "sacchetto giuliano                                                                                                                                                                                       2230\n",
      "Falcom Sound Team jdk                                                                                                                                                                                    2167\n",
      "RR                                                                                                                                                                                                       2059\n",
      "Pyotr Ilyich Tchaikovsky                                                                                                                                                                                 2045\n",
      "Claude Debussy                                                                                                                                                                                           1950\n",
      "Giordano Trivellato| sacchetto giuliano                                                                                                                                                                  1927\n",
      "Cole Porter                                                                                                                                                                                              1916\n",
      "Rights Reserved                                                                                                                                                                                          1873\n",
      "Irving Berlin                                                                                                                                                                                            1753\n",
      "George Frideric Handel                                                                                                                                                                                   1672\n",
      "Giuseppe Verdi                                                                                                                                                                                           1655\n",
      "Bob Dylan                                                                                                                                                                                                1562\n",
      "Giacomo Puccini                                                                                                                                                                                          1519\n",
      "Ludwig Van Beethoven                                                                                                                                                                                     1492\n",
      "Felix Mendelssohn                                                                                                                                                                                        1319\n",
      "                                                                                                                                                                                                        ...  \n",
      "Mike Stevens| Aadil Rasheed| Mark Wynter                                                                                                                                                                    1\n",
      "Mike Viola| James Flannigan| Andrew McMahon                                                                                                                                                                 1\n",
      "Mike Walker| Jeff Middleton| Austin Jenkes                                                                                                                                                                  1\n",
      "MikeFreedom                                                                                                                                                                                                 1\n",
      "Miki Hernandez                                                                                                                                                                                              1\n",
      "Mike Batt| Katei Melua                                                                                                                                                                                      1\n",
      "Mikkel Eriksen| Tor Erik Hermansen| Sandy Julien Wilhelm                                                                                                                                                    1\n",
      "Mikkel Storleer Eriksen| Tor Erik Hermansen| Christopher Anthony John Martin| Guy Rupert Berryman| William Champion| Jonathan Mark Buckland| Venor Timothy Yard| Marcos Jesus Tovar| Scott Alan Zant        1\n",
      "Mikkey Vall| Danielmusic                                                                                                                                                                                    1\n",
      "Miko/Yumi                                                                                                                                                                                                   1\n",
      "Mikrousov| Line Adam                                                                                                                                                                                        1\n",
      "Miles Davis| M. Davis                                                                                                                                                                                       1\n",
      "Mike Percy| John McLaughlin| Tim Lever| Jimmy Constable                                                                                                                                                     1\n",
      "Mike Park                                                                                                                                                                                                   1\n",
      "Mike Parish                                                                                                                                                                                                 1\n",
      "Mike Mogis| Nathaniel Walcott| Stefanie Drootin-Senseney| Chris Senseney                                                                                                                                    1\n",
      "Mike McCready                                                                                                                                                                                               1\n",
      "Mike Martin                                                                                                                                                                                                 1\n",
      "Mike Mangini|Bruce Hornsby|Bob Dylan                                                                                                                                                                        1\n",
      "Mike L. Williams Ii| Pierre Ramon Slaughter| Timothy Thomas| Theron Thomas| Miley Cyrus| Douglas Davis| Ricky Walters                                                                                       1\n",
      "Mike Knott                                                                                                                                                                                                  1\n",
      "Mike Kintish| Andrew Bullimore| Sam O'Neill                                                                                                                                                                 1\n",
      "Mike Henderson|Tammy Rogers                                                                                                                                                                                 1\n",
      "Mike E Clark| Violent J                                                                                                                                                                                     1\n",
      "Mike Donehey|Jeff Owen|Ruben Juarez                                                                                                                                                                         1\n",
      "Mike Dallon                                                                                                                                                                                                 1\n",
      "Mike D'Abo                                                                                                                                                                                                  1\n",
      "Mike Curb| Guy Hemric                                                                                                                                                                                       1\n",
      "Mike City| Tierra Brown                                                                                                                                                                                     1\n",
      "\u0003伍樂城                                                                                                                                                                                                        1\n",
      "Name: composer, Length: 329823, dtype: int64\n",
      "\n",
      "lyricist属性有110926的不同取值，各取值及其出现的次数\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Traditional                                                                                                                                                                                                                                                       1751\n",
      "―                                                                                                                                                                                                                                                                 1530\n",
      "林夕                                                                                                                                                                                                                                                                1015\n",
      "Michael Ruland                                                                                                                                                                                                                                                     832\n",
      "Lin Xi                                                                                                                                                                                                                                                             671\n",
      "Dieter Bohlen                                                                                                                                                                                                                                                      660\n",
      "李岩修                                                                                                                                                                                                                                                                633\n",
      "黃偉文                                                                                                                                                                                                                                                                626\n",
      "姚若龍                                                                                                                                                                                                                                                                529\n",
      "幼福文化                                                                                                                                                                                                                                                               481\n",
      "Bob Dylan                                                                                                                                                                                                                                                          441\n",
      "ณรงค์วิทย์ เตชะธนะวัฒน์                                                                                                                                                                                                                                            434\n",
      "สีฟ้า                                                                                                                                                                                                                                                              432\n",
      "Akimoto Yasushi                                                                                                                                                                                                                                                    418\n",
      "ฟองเบียร์                                                                                                                                                                                                                                                          397\n",
      "สลา คุณวุฒิ                                                                                                                                                                                                                                                        389\n",
      "inst                                                                                                                                                                                                                                                               374\n",
      "秋元 康                                                                                                                                                                                                                                                               362\n",
      "B. Dylan                                                                                                                                                                                                                                                           361\n",
      "R. Kelly                                                                                                                                                                                                                                                           357\n",
      "Noel Gallagher                                                                                                                                                                                                                                                     351\n",
      "Leung May May                                                                                                                                                                                                                                                      342\n",
      "方文山                                                                                                                                                                                                                                                                339\n",
      "Richard Lam Chun Keung                                                                                                                                                                                                                                             331\n",
      "TSUNKU                                                                                                                                                                                                                                                             330\n",
      "John Denver                                                                                                                                                                                                                                                        320\n",
      "阿信                                                                                                                                                                                                                                                                 314\n",
      "เสกสรรค์ ศุขพิมาย                                                                                                                                                                                                                                                  308\n",
      "周耀輝                                                                                                                                                                                                                                                                300\n",
      "新韻傳音                                                                                                                                                                                                                                                               295\n",
      "                                                                                                                                                                                                                                                                  ... \n",
      "Michael Stevenson|Ryan Toby|Lundon|Kwamé                                                                                                                                                                                                                             1\n",
      "Michael Sembello| Zhen Qiang Lin                                                                                                                                                                                                                                     1\n",
      "Michael Schaller| Thomas H. Schaller                                                                                                                                                                                                                                 1\n",
      "While Chris                                                                                                                                                                                                                                                          1\n",
      "Will Smith| Lee Haggard| Wayne Williams| Keith Mayberry| Gregory Webster| Marvin Pierce| Norman Napier| Walter Morrison| Ralph Middlebrooks| Marshall Jones| Leroy Bonner| Andrew Noland                                                                             1\n",
      "Will Smith| Teddy Riley| Menton Smith                                                                                                                                                                                                                                1\n",
      "Michael Omartian                                                                                                                                                                                                                                                     1\n",
      "Will Smith| Zane Grey| Len Ron Hanks| Kim Jones| Lennie Bennett                                                                                                                                                                                                      1\n",
      "William Barnes                                                                                                                                                                                                                                                       1\n",
      "Michael McHenry|Ryan Buendia|Alexander Ridha|Jean-Baptiste Kouame|Natassia Zolot|Jonas Jeberg                                                                                                                                                                        1\n",
      "William Masson                                                                                                                                                                                                                                                       1\n",
      "Michael Masser| Linda Creed| Shep Crawford                                                                                                                                                                                                                           1\n",
      "Michael Luna|Grace Slick|Jeff Pennig                                                                                                                                                                                                                                 1\n",
      "Michael Kunze|Helmut Frey                                                                                                                                                                                                                                            1\n",
      "Won Tae Teon| Wu April| Yu Julia                                                                                                                                                                                                                                     1\n",
      "Willie Hale                                                                                                                                                                                                                                                          1\n",
      "Wilton Vought (BMI)                                                                                                                                                                                                                                                  1\n",
      "Michael Jones| D. Letts                                                                                                                                                                                                                                              1\n",
      "Michael Jackson|Rodney Jerkins|Fred Jerkins III|LaShawn Daniels                                                                                                                                                                                                      1\n",
      "Michael Jackson| Teddy Riley                                                                                                                                                                                                                                         1\n",
      "Michael Jackson| See Sub-songs| James Samuel Harris III| Rodney Jerkins| Terry Lewis| Fred Jerkins III| Bernard Belle| LaShawn Daniels                                                                                                                               1\n",
      "Michael Jackson| Rockwell| See Sub-songs| Eddie Cascio| James Samuel Harris III| Rodney Jerkins| Teddy Riley| Terry Lewis| Fred Jerkins III| James Porte| Curtis Jackson| LaShawn Daniels                                                                            1\n",
      "Michael Jackson| Janet Jackson| See Sub-songs| Terry Lewis| James Samuel Harris III                                                                                                                                                                                  1\n",
      "Michael Jackson| Janet Jackson| Berry Gordy| Jr.| Freddie Perren| Hal Davis| Jimmie Thomas| See Sub-songs| Teddy Riley| Alphonso J. Mizell| Bernard Belle| Deke Richards| Rodney Jerkins| Terry Lewis| Bill Bottrell| Bruce Swedien| Bob West| Fred Jerkins| J       1\n",
      "Michael Jackson+鄭楠                                                                                                                                                                                                                                                   1\n",
      "Witold Mikolajczuk                                                                                                                                                                                                                                                   1\n",
      "Woddy Guthrie                                                                                                                                                                                                                                                        1\n",
      "Wolfgang Lenk|Mathias Dietrich|Alexander Zieme|Sebastian Krumbiegel|Tobias Künzel|Henri Schmidt|Jens Sembdner                                                                                                                                                        1\n",
      "Michael Grundmann                                                                                                                                                                                                                                                    1\n",
      "ＺＯＰＰ                                                                                                                                                                                                                                                                 1\n",
      "Name: lyricist, Length: 110925, dtype: int64\n",
      "\n",
      "language属性有11的不同取值，各取值及其出现的次数\n",
      "\n",
      "52.0    1336694\n",
      "-1.0     639467\n",
      "3.0      106295\n",
      "17.0      92518\n",
      "24.0      41744\n",
      "31.0      39201\n",
      "10.0      15482\n",
      "45.0      14435\n",
      "59.0       8098\n",
      "38.0       2385\n",
      "Name: language, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "for col in list(songs.columns):\n",
    "    num_vlaules = len(songs[col].unique())\n",
    "    print('\\n%s属性有%d的不同取值，各取值及其出现的次数\\n'% (col,num_vlaules))\n",
    "    print(songs[col].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "genre_ids，composer存在多个用'|'分割情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "song_id属性有2295971的不同取值，各取值及其出现的次数\n",
      "\n",
      "00CK1AeW3NdWAxhuPfe1r5x8545m+pg0MTI3Q7eHCYQ=    1\n",
      "jbpBFLMVO8yvNA2NO0fxYjbsWmLZUn+xWFnt1nRt4n4=    1\n",
      "izrNQSPrtErsHFcIsvMWhGmkM3SY/15fUwy5QHH3dRY=    1\n",
      "lLJSNVfr9RjccfdUGk/cVW6uKJIu85xDGuWtPtFkwdw=    1\n",
      "oKPZ0Bl66Yxd9iim88Z81c4GLzF4DZ9+3exZxj5a+zo=    1\n",
      "qct0LADT9yWCV1xzvE9BpDKInSxlBK0JhogqOHQEn+c=    1\n",
      "rxwoaxIAGeyZZLREK6QRW7RNnEOZPJXv1+PAkRbO1vI=    1\n",
      "fuW81gbEaGKPxlJ1bqP4PU+6mNZlPp570dX7qrzFRcE=    1\n",
      "c4PokSTKW1wxBEpGJD5Ep3p7T3eBToiDUaNTcYwEuEE=    1\n",
      "ZNZbw5IoGbkRZJk3W/53PHXMiWpWgmJdeTcrgrvQgdU=    1\n",
      "gTR7Dk1jdag3vG7ML3G95q9+J/T0NlDfV+WxPACsXqQ=    1\n",
      "venQtZyn1hcXzI3Ii5O/3eFnGocc4h9JoxC4jb3+bLg=    1\n",
      "FU8XwQNS5fagw+Qq+5IoTyvnqVtKbDrTjbD+kyenqqU=    1\n",
      "tf0ijmnYovaoF9FmRX2zCCLU7KMLdH5ziBjgRz7+klQ=    1\n",
      "X9cuxp+7dLQ2Z3uzSxTfsS1oDuL4uvJ+32OdJZmJKu4=    1\n",
      "4RFnTBy/xJU6Ohn3LaJ58FbY5N0sjZ2+Lvho4/iYoxU=    1\n",
      "CKutmS9JPDe6+/pg+CGrpzxoMpqCofN/lqMsGltS34Y=    1\n",
      "aidJKy1q+j/suubQoZlyWUDSbEpp519XoaQqgN8QWLk=    1\n",
      "yW3WryUEvGx7F+TZgyVMv12vf4X5JYNSQC7X4wLemtI=    1\n",
      "x22UJw6rcxGwuGO4cHcJYdqT3UwKS6KQHgPJtplZKXg=    1\n",
      "fcX0RxktBcVV/XrDs/0rPqU7RP1f1yztfJG+vIfPYkk=    1\n",
      "MblQjz0+nqUlgQP4FMOch8HYK86AkHy2Y0kJyB630pI=    1\n",
      "bOSl7MXNBdm5yq56bZO7O5o4fiLISZ7HlKqaKYsH1wc=    1\n",
      "N3LtObWyhWfdQsvyCRJ60zxgBeCp7GkVk8ygUFVnRuE=    1\n",
      "Ix6a4I7QrhkY8RA6+6rIDNVEpEkd+0V1Xi9EpczKhq4=    1\n",
      "hr35qymI/wKCQ+pdCQkmbNOpDx2T0H35aIDP3iXcLTk=    1\n",
      "brPT9qM5C5E52OKBL88Vu/iFciDp+vd2MuWGwyVzuS4=    1\n",
      "m2oiZZup6QpuqlXWZ4sHd8FHok3uJjbiHkMYG1zkixs=    1\n",
      "/m25Qnf0NTd6cHEe6kDTv710HCUtD2RULMQyOwKo96s=    1\n",
      "s+MwjeYOSFVlKOjK0MJIMsFZq1krITq57Ej7kdHho5g=    1\n",
      "                                               ..\n",
      "0VMTeUXFdInx7oVJ5yh44MZgiD8NY2TCSPB1jupDqQU=    1\n",
      "EvsPGGddEnntnbiQfc4EEfIBBNNoUjo43pNEeTUu3wI=    1\n",
      "oXUM9MrpKm+4nrTiJb8tRn1hVEakVMchcnx7BJpa8p8=    1\n",
      "Y7jxGrdW38/rLQtby16+qbOVXcDVDP7FLfkAVgL/vn8=    1\n",
      "keddbLqv7WtTXnFmRelEV9aCglNmqFL9iLZAvp96Ukk=    1\n",
      "SR3Sabw6Ph72Dfo4jMbRu8ev5/wLz+poHTH7cekfDXM=    1\n",
      "U8u1+MEn+4guUua1+A1C/NIqzP6zt8ho1r3b37UaTKU=    1\n",
      "uRRAmVdcZXS0pWajmB+kVJiF/Uao0erTTM7GJtFUpww=    1\n",
      "q707Gc+5kU0Zrcoh82pb/41mbgN2m4UjGRA34bdX340=    1\n",
      "zv8dGFhyrIbaVUK+MT09VYb6rzxhHyiC1zKbVQTFbAk=    1\n",
      "HnAd5vbRZ1rBd1+M8MbWHcNuuvJqdQMh0qEJxSNIcYU=    1\n",
      "dw5WGKVSaEAxDkA+K36JJ32iOJysMsvVXIdYBaaAtzs=    1\n",
      "Xfg/znr3SOMzSRlPjvIEfac5p+zflw7XPmA9NHWvJMo=    1\n",
      "y9H1EcNfYVE/S4B2Us9NIavfGyMhdsbXC1ON7UqiXl8=    1\n",
      "1tFH4wMT2LQn9Bhn/R6XJ1WKxf8gwImPybb8L+l2evc=    1\n",
      "laQqx4A2+Gb4htga29YiB1XxS8HrtUP98j0KMXF8/T0=    1\n",
      "KRi1VIowcE7qYb5ix8xbIvshtxVymHq8aS5+aOnUEhU=    1\n",
      "7eGTE4igfWDJuF/NNoS4kSJFp1PUOucmnoyMC3tC64k=    1\n",
      "1UnRIgIGy9i6EpEKCtHg4ZpRjmjK/FpXRQVuj7JjkAU=    1\n",
      "0IDo0YDziW9Tw01VzuMFkDOREEpVE8Z8hZ85Lak+SLY=    1\n",
      "tH5Ux2i4RCVmEkx/OfDIEXx6emjbNb15tMNZT1KLnII=    1\n",
      "/39qk7Zb/A18sk9rndaJqd9TAmdf2XaW/5aSUDxg3zs=    1\n",
      "LAMcOEKyFUEsbsDurgVLSbsOFqhcrKghrV+hSDLbGY8=    1\n",
      "F7/e48wvjbyQnSSt0AlFZuHBd7jDnYo7NOmz9L5Woks=    1\n",
      "SuYGDo8S6zsRiYdCYCQWZspwDHCwQKB2X/iNaBpFJ3E=    1\n",
      "6ROHz0fkDobBk7TG+c6cRuJzCnkybcW6KwFApGhpj90=    1\n",
      "ThxHlzaL1t4zUHzTcIdQBQEr5RwJuOM1Jgw3z81NNAE=    1\n",
      "RkqAkpgGHU9WyCW4+YhswD06Nfs+OhQhwpEC6YU+gAo=    1\n",
      "7gY5yh6D4SutIPje7sHD1sRQtH6Tga7CAd92AsjYy6A=    1\n",
      "YoLL24fey4indCX71pFtH3Q91Qm7gCeB5M7XS65daSs=    1\n",
      "Name: song_id, Length: 2295971, dtype: int64\n",
      "\n",
      "name属性有1168980的不同取值，各取值及其出现的次数\n",
      "\n",
      "Intro                                                                          1734\n",
      "Silent Night                                                                   1088\n",
      "White Christmas                                                                1018\n",
      "Jingle Bells                                                                    964\n",
      "Ave Maria                                                                       792\n",
      "Summertime                                                                      762\n",
      "Amazing Grace                                                                   724\n",
      "1. Allegro                                                                      688\n",
      "Home                                                                            677\n",
      "Winter Wonderland                                                               596\n",
      "Moon River                                                                      576\n",
      "Time                                                                            557\n",
      "Autumn Leaves                                                                   556\n",
      "The First Noel                                                                  512\n",
      "O Holy Night                                                                    507\n",
      "Stay                                                                            491\n",
      "Without You                                                                     485\n",
      "Tonight                                                                         480\n",
      "Unchained Melody                                                                480\n",
      "The Christmas Song                                                              475\n",
      "Hello                                                                           475\n",
      "Overture                                                                        454\n",
      "You                                                                             451\n",
      "Smile                                                                           445\n",
      "I Love You                                                                      445\n",
      "Have Yourself a Merry Little Christmas                                          443\n",
      "I. Allegro                                                                      439\n",
      "Last Christmas                                                                  432\n",
      "Sleigh Ride                                                                     416\n",
      "Hallelujah                                                                      415\n",
      "                                                                               ... \n",
      "Ich weiss nicht was ich will                                                      1\n",
      "Memes| it seems                                                                   1\n",
      "Una Oportunidad                                                                   1\n",
      "Invitatorium Hodie Exultandum                                                     1\n",
      "Hardstyle's Paradise (Dj Activator Rmx)                                           1\n",
      "I’m Your Dog                                                                      1\n",
      "Valses Poéticos: IV. Vals lento| Tiempo de vals lento                             1\n",
      "爱走远                                                                               1\n",
      "A Quiet Pain                                                                      1\n",
      "VERSACE PYTHON                                                                    1\n",
      "Jetè 1                                                                            1\n",
      "隨風歸去                                                                              1\n",
      "Contrasts| Sz. 111: III. Sebes                                                    1\n",
      "Potatoes Being Peeled Manually                                                    1\n",
      "男人二十七                                                                             1\n",
      "Dosimeter-feat. Nick Thayer                                                       1\n",
      "Sì di ritrovarla io giuro                                                         1\n",
      "Wolken                                                                            1\n",
      "Sunset Moonrise                                                                   1\n",
      "Cheezy(Katy's Kool)                                                               1\n",
      "Dim Mak Day Trip                                                                  1\n",
      "Fauré: Pavane| Op. 50                                                             1\n",
      "Red Lights (2017 Dance Workout Mix)                                               1\n",
      "GOVERNMENT CHEESE                                                                 1\n",
      "Talkin Back                                                                       1\n",
      "Me Quedo                                                                          1\n",
      "禮拜天情人                                                                             1\n",
      "有緣來做伙                                                                             1\n",
      "Gran Duetto| Op. 87: I. Maestoso                                                  1\n",
      "Jelek| játékok és üzenetek / Signs| Games and Messages: Jelek II / Signs II       1\n",
      "Name: name, Length: 1168979, dtype: int64\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "isrc属性有1806826的不同取值，各取值及其出现的次数\n",
      "\n",
      "GBPS81518952    207\n",
      "GBPS81527416    205\n",
      "GBPS81518950    195\n",
      "GBPS81518680    195\n",
      "GBPS81518682    194\n",
      "GBPS81527413    193\n",
      "GBPS81527817    189\n",
      "GBPS81527820    186\n",
      "GBPS81527823    184\n",
      "GBPS81518948    183\n",
      "GBPS81518947    174\n",
      "GBPS81518943    174\n",
      "GBPS81527417    173\n",
      "GBPS81518945    172\n",
      "GBPS81527822    170\n",
      "US6R21419985    167\n",
      "GBPS81518944    154\n",
      "GBPS81400018    145\n",
      "GBPS81518681    132\n",
      "GBPS81529212    131\n",
      "GBPS81528923    129\n",
      "GBPS81529178    129\n",
      "GBPS81529179    128\n",
      "USV351456233    127\n",
      "GBPS81529213    127\n",
      "USV291427155    126\n",
      "GBPS81528967    126\n",
      "GBPS81529089    126\n",
      "GBPS81527818    126\n",
      "GBPS81528929    126\n",
      "               ... \n",
      "USVWC1102101      1\n",
      "GBUM71602889      1\n",
      "USUM71403086      1\n",
      "USTCE1068002      1\n",
      "US4R31148539      1\n",
      "PLL431466739      1\n",
      "JPG200607479      1\n",
      "TCACO1686513      1\n",
      "DEN961531263      1\n",
      "USPJT1100010      1\n",
      "HKUM71000066      1\n",
      "DEPZ68817730      1\n",
      "FR6P11500350      1\n",
      "JPQ700200060      1\n",
      "GB5P51500071      1\n",
      "GB32E1403615      1\n",
      "USWH10500027      1\n",
      "USUM71414304      1\n",
      "FR0W61138468      1\n",
      "DEUM71503605      1\n",
      "USWB19900686      1\n",
      "ES7031000106      1\n",
      "DK4YA1207801      1\n",
      "USWMA9022128      1\n",
      "FR7JQ1600050      1\n",
      "FR6V82210995      1\n",
      "TWF750688204      1\n",
      "JPN351601787      1\n",
      "US3DF1534029      1\n",
      "DEKB71509650      1\n",
      "Name: isrc, Length: 1806825, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "for col in list(songs_extra.columns):\n",
    "    num_vlaules = len(songs_extra[col].unique())\n",
    "    print('\\n%s属性有%d的不同取值，各取值及其出现的次数\\n'% (col,num_vlaules))\n",
    "    print(songs_extra[col].value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#训练数据测试数据与歌曲库关联\n",
    "train = train.merge(songs, on='song_id', how='left')\n",
    "test = test.merge(songs, on='song_id', how='left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#从registration_init_time和expiration_date 提取年月日两个特征相差天数，作为新的特征，删除原有的特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "members['membership_days'] = members['expiration_date'].subtract(members['registration_init_time']).dt.days.astype(int)\n",
    "members['registration_year'] = members['registration_init_time'].dt.year\n",
    "members['registration_month'] = members['registration_init_time'].dt.month\n",
    "members['registration_date'] = members['registration_init_time'].dt.day\n",
    "members['expiration_year'] = members['expiration_date'].dt.year\n",
    "members['expiration_month'] = members['expiration_date'].dt.month\n",
    "members['expiration_date'] = members['expiration_date'].dt.day\n",
    "members = members.drop(['registration_init_time'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#从isrc中提取歌曲时间年"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def isrc_to_year(isrc):\n",
    "    if type(isrc) == str:\n",
    "        if int(isrc[5:7]) > 17:\n",
    "            return 1900 + int(isrc[5:7])\n",
    "        else:\n",
    "            return 2000 + int(isrc[5:7])\n",
    "    else:\n",
    "        return np.nan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "songs_extra['song_year'] = songs_extra['isrc'].apply(isrc_to_year)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#删除歌曲isrc, name特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "songs_extra.drop(['isrc', 'name'], axis = 1, inplace = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#训练数据测试数据与用户数据关联"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train.merge(members, on='msno', how='left')\n",
    "test = test.merge(members, on='msno', how='left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#训练数据测试数据与歌曲扩展数据关联"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train.merge(songs_extra, on = 'song_id', how = 'left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#歌曲长度为空的数据用200000填补并进行类别转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.song_length.fillna(200000,inplace=True)\n",
    "train.song_length = train.song_length.astype(np.uint32)\n",
    "train.song_id = train.song_id.astype('category')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#测试数据同样处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "test = test.merge(songs_extra, on = 'song_id', how = 'left')\n",
    "test.song_length.fillna(200000,inplace=True)\n",
    "test.song_length = test.song_length.astype(np.uint32)\n",
    "test.song_id = test.song_id.astype('category')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#增加一个新的特征genre_ids带有|id个数进行计数操作\n",
    "def genre_id_count(x):\n",
    "    if x == 'no_genre_id':\n",
    "        return 0\n",
    "    else:\n",
    "        return x.count('|') + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "#genre_ids中添加一个新的类别no_genre_id\n",
    "train['genre_ids']=train['genre_ids'].cat.add_categories(['no_genre_id']);\n",
    "test['genre_ids']=test['genre_ids'].cat.add_categories(['no_genre_id']);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "#genre_ids为空的数据使用no_genre_id类别替换\n",
    "train['genre_ids'].fillna('no_genre_id',inplace=True)\n",
    "test['genre_ids'].fillna('no_genre_id',inplace=True)\n",
    "#多个id的进行计数加入genre_ids_count新特征重\n",
    "train['genre_ids_count'] = train['genre_ids'].apply(genre_id_count).astype(np.int8)\n",
    "test['genre_ids_count'] = test['genre_ids'].apply(genre_id_count).astype(np.int8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def composer_count(x):\n",
    "    if x == 'no_composer':\n",
    "        return 0\n",
    "    else:\n",
    "        return sum(map(x.count, ['|', '/', '\\\\', ';'])) + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#composer中添加一个新的类别no_composer\n",
    "train['composer']=train['composer'].cat.add_categories(['no_composer']);\n",
    "test['composer']=test['composer'].cat.add_categories(['no_composer']);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "#composer为空的数据使用no_composer类别替换\n",
    "train['composer'].fillna('no_composer',inplace=True)\n",
    "test['composer'].fillna('no_composer',inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "#对composer计数 构成一个新的特征composer_count\n",
    "train['composer_count'] = train['composer'].apply(composer_count).astype(np.int8)\n",
    "test['composer_count'] = test['composer'].apply(composer_count).astype(np.int8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_featured(x):\n",
    "    if 'feat' in str(x) :\n",
    "        return 1\n",
    "    return 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#对artist_name空值填补，对artist_name特征类别分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['artist_name']=train['artist_name'].cat.add_categories(['no_artist']);\n",
    "test['artist_name']=test['artist_name'].cat.add_categories(['no_artist']);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['artist_name'].fillna('no_artist',inplace=True)\n",
    "test['artist_name'].fillna('no_artist',inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['is_featured'] = train['artist_name'].apply(is_featured).astype(np.int8)\n",
    "test['is_featured'] = test['artist_name'].apply(is_featured).astype(np.int8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def artist_count(x):\n",
    "    if x == 'no_artist':\n",
    "        return 0\n",
    "    else:\n",
    "        return x.count('and') + x.count(',') + x.count('feat') + x.count('&')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['artist_count'] = train['artist_name'].apply(artist_count).astype(np.int8)\n",
    "test['artist_count'] = test['artist_name'].apply(artist_count).astype(np.int8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 判断艺术家和作曲家是一样，如果一样放入artist_composer特征\n",
    "train['artist_name']=train['artist_name'].astype(object)\n",
    "train['composer']=train['composer'].astype(object)\n",
    "test['artist_name']=test['artist_name'].astype(object)\n",
    "test['composer']=test['composer'].astype(object)\n",
    "train['artist_composer'] =(train['artist_name'] ==train['composer']).astype(np.int8)\n",
    "test['artist_composer'] = (test['artist_name'] == test['composer']).astype(np.int8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 如果艺术家、抒情家和作曲家都是一样的 放入新的特征\n",
    "train['lyricist']=train['lyricist'].astype(object)\n",
    "test['lyricist']=test['lyricist'].astype(object)\n",
    "train['artist_composer_lyricist'] = np.array((train['artist_name'] is train['composer']) & (train['artist_name'] is train['lyricist']) & (train['composer'] is train['lyricist'])).astype(np.int8)\n",
    "test['artist_composer_lyricist'] = np.array((test['artist_name'] is test['composer']) & (test['artist_name'] is test['lyricist']) & (test['composer'] is test['lyricist'])).astype(np.int8)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 判断歌曲语言是否是 17 or 45. \n",
    "def song_lang_boolean(x):\n",
    "    if '17.0' in str(x) or '45.0' in str(x):\n",
    "        return 1\n",
    "    return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "#将为空的数据填充为0\n",
    "train['language']=train['language'].replace(np.NaN,0)\n",
    "test['language']=test['language'].replace(np.NaN,0)\n",
    "train['song_lang_boolean'] = train['language'].apply(song_lang_boolean).astype(np.int8)\n",
    "test['song_lang_boolean'] = test['language'].apply(song_lang_boolean).astype(np.int8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "_mean_song_length = np.mean(train['song_length'])\n",
    "def smaller_song(x):\n",
    "    if x < _mean_song_length:\n",
    "        return 1\n",
    "    return 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#将歌曲长度小于平均值的提取出来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['smaller_song'] = train['song_length'].apply(smaller_song).astype(np.int8)\n",
    "test['smaller_song'] = test['song_length'].apply(smaller_song).astype(np.int8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#播放过歌曲的次数提取出来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "_dict_count_song_played_train = {k: v for k, v in train['song_id'].value_counts().iteritems()}\n",
    "_dict_count_song_played_test = {k: v for k, v in test['song_id'].value_counts().iteritems()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_song_played(x):\n",
    "    try:\n",
    "        return _dict_count_song_played_train[x]\n",
    "    except KeyError:\n",
    "        try:\n",
    "            return _dict_count_song_played_test[x]\n",
    "        except KeyError:\n",
    "            return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['count_song_played'] = train['song_id'].apply(count_song_played).astype(np.int64)\n",
    "test['count_song_played'] = test['song_id'].apply(count_song_played).astype(np.int64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#播放艺术家的次数提取出来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of times the artist has been played\n",
    "_dict_count_artist_played_train = {k: v for k, v in train['artist_name'].value_counts().iteritems()}\n",
    "_dict_count_artist_played_test = {k: v for k, v in test['artist_name'].value_counts().iteritems()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_artist_played(x):\n",
    "    try:\n",
    "        return _dict_count_artist_played_train[x]\n",
    "    except KeyError:\n",
    "        try:\n",
    "            return _dict_count_artist_played_test[x]\n",
    "        except KeyError:\n",
    "            return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "train['count_artist_played'] = train['artist_name'].apply(count_artist_played).astype(np.int64)\n",
    "test['count_artist_played'] = test['artist_name'].apply(count_artist_played).astype(np.int64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "gender、source_system_tab、source_screen_name、source_type空值填补"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['gender']=train['gender'].cat.add_categories(['Unknown']);\n",
    "test['gender']=test['gender'].cat.add_categories(['Unknown']);\n",
    "train['source_system_tab']=train['source_system_tab'].cat.add_categories(['Unknown']);\n",
    "test['source_system_tab']=test['source_system_tab'].cat.add_categories(['Unknown']);\n",
    "train['source_type']=train['source_type'].cat.add_categories(['Unknown']);\n",
    "test['source_type']=test['source_type'].cat.add_categories(['Unknown']);\n",
    "train['gender'].fillna(value=\"Unknown\",inplace=True)\n",
    "test['gender'].fillna(value=\"Unknown\",inplace=True)\n",
    "train['source_system_tab'].fillna(value=\"Unknown\",inplace=True)\n",
    "test['source_system_tab'].fillna(value=\"Unknown\",inplace=True)\n",
    "train['source_screen_name'].fillna(value=\"Unknown\",inplace=True)\n",
    "test['source_screen_name'].fillna(value=\"Unknown\",inplace=True)\n",
    "train['source_type'].fillna(value=\"Unknown\",inplace=True)\n",
    "test['source_type'].fillna(value=\"Unknown\",inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#年龄分类提取提取新特征age_category"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "train['age_range'] = pd.cut(train['bd'],bins=[-45,0,10,18,35,50,80,1052])\n",
    "test['age_range'] = pd.cut(test['bd'],bins=[-45,0,10,18,35,50,80,1052])\n",
    "\n",
    "combine = [train, test]\n",
    "\n",
    "for dataset in combine : \n",
    "    \n",
    "    dataset.loc[(dataset['bd'] > 0) & (dataset['bd'] <= 10), 'age_category'] = 0\n",
    "    dataset.loc[(dataset['bd'] > 80) & (dataset['bd'] <= 1052), 'age_category'] = 1\n",
    "    dataset.loc[(dataset['bd'] > 50) & (dataset['bd'] <= 80), 'age_category'] = 2\n",
    "    dataset.loc[(dataset['bd'] > 10) & (dataset['bd'] <= 18), 'age_category'] = 3\n",
    "    dataset.loc[(dataset['bd'] > 35) & (dataset['bd'] <= 50), 'age_category'] = 4\n",
    "    dataset.loc[(dataset['bd'] > -45) & (dataset['bd'] <= 0), 'age_category'] = 5\n",
    "    dataset.loc[(dataset['bd'] > 18) & (dataset['bd'] <= 35), 'age_category'] = 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.drop([\"age_range\",\"bd\"],axis=1,inplace=True)\n",
    "test.drop([\"age_range\",\"bd\"],axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>song_id</th>\n",
       "      <th>source_system_tab</th>\n",
       "      <th>source_screen_name</th>\n",
       "      <th>source_type</th>\n",
       "      <th>target</th>\n",
       "      <th>song_length</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>composer</th>\n",
       "      <th>...</th>\n",
       "      <th>composer_count</th>\n",
       "      <th>is_featured</th>\n",
       "      <th>artist_count</th>\n",
       "      <th>artist_composer</th>\n",
       "      <th>artist_composer_lyricist</th>\n",
       "      <th>song_lang_boolean</th>\n",
       "      <th>smaller_song</th>\n",
       "      <th>count_song_played</th>\n",
       "      <th>count_artist_played</th>\n",
       "      <th>age_category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=</td>\n",
       "      <td>BBzumQNXUHKdEBOB7mAJuzok+IJA1c2Ryg/yzTF6tik=</td>\n",
       "      <td>explore</td>\n",
       "      <td>Explore</td>\n",
       "      <td>online-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>206471</td>\n",
       "      <td>359</td>\n",
       "      <td>Bastille</td>\n",
       "      <td>Dan Smith| Mark Crew</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>215</td>\n",
       "      <td>1140</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>bhp/MpSNoqoxOIB+/l8WPqu6jldth4DIpCm3ayXnJqM=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>284584</td>\n",
       "      <td>1259</td>\n",
       "      <td>Various Artists</td>\n",
       "      <td>no_composer</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>303616</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>JNWfrrC7zNN7BdMpsISKa4Mw+xVJYNnxXh3/Epw7QgY=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>225396</td>\n",
       "      <td>1259</td>\n",
       "      <td>Nas</td>\n",
       "      <td>N. Jones、W. Adams、J. Lordan、D. Ingle</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>289</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=</td>\n",
       "      <td>2A87tzfnJTSWqD7gIZHisolhe4DMdzkbd6LzO1KHjNs=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>255512</td>\n",
       "      <td>1019</td>\n",
       "      <td>Soundway</td>\n",
       "      <td>Kwadwo Donkoh</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=</td>\n",
       "      <td>3qm6XTZ6MOCU11x8FIVbAGH5l5uMkT3/ZalWG1oo2Gc=</td>\n",
       "      <td>explore</td>\n",
       "      <td>Explore</td>\n",
       "      <td>online-playlist</td>\n",
       "      <td>1</td>\n",
       "      <td>187802</td>\n",
       "      <td>1011</td>\n",
       "      <td>Brett Young</td>\n",
       "      <td>Brett Young| Kelly Archer| Justin Ebach</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>412</td>\n",
       "      <td>427</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno  \\\n",
       "0  FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=   \n",
       "1  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "2  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "3  Xumu+NIjS6QYVxDS4/t3SawvJ7viT9hPKXmf0RtLNx8=   \n",
       "4  FGtllVqz18RPiwJj/edr2gV78zirAiY/9SmYvia+kCg=   \n",
       "\n",
       "                                        song_id source_system_tab  \\\n",
       "0  BBzumQNXUHKdEBOB7mAJuzok+IJA1c2Ryg/yzTF6tik=           explore   \n",
       "1  bhp/MpSNoqoxOIB+/l8WPqu6jldth4DIpCm3ayXnJqM=        my library   \n",
       "2  JNWfrrC7zNN7BdMpsISKa4Mw+xVJYNnxXh3/Epw7QgY=        my library   \n",
       "3  2A87tzfnJTSWqD7gIZHisolhe4DMdzkbd6LzO1KHjNs=        my library   \n",
       "4  3qm6XTZ6MOCU11x8FIVbAGH5l5uMkT3/ZalWG1oo2Gc=           explore   \n",
       "\n",
       "    source_screen_name      source_type  target  song_length genre_ids  \\\n",
       "0              Explore  online-playlist       1       206471       359   \n",
       "1  Local playlist more   local-playlist       1       284584      1259   \n",
       "2  Local playlist more   local-playlist       1       225396      1259   \n",
       "3  Local playlist more   local-playlist       1       255512      1019   \n",
       "4              Explore  online-playlist       1       187802      1011   \n",
       "\n",
       "       artist_name                                 composer  ...  \\\n",
       "0         Bastille                     Dan Smith| Mark Crew  ...   \n",
       "1  Various Artists                              no_composer  ...   \n",
       "2              Nas     N. Jones、W. Adams、J. Lordan、D. Ingle  ...   \n",
       "3         Soundway                            Kwadwo Donkoh  ...   \n",
       "4      Brett Young  Brett Young| Kelly Archer| Justin Ebach  ...   \n",
       "\n",
       "  composer_count is_featured artist_count artist_composer  \\\n",
       "0              2           0            0               0   \n",
       "1              0           0            0               0   \n",
       "2              1           0            0               0   \n",
       "3              1           0            0               0   \n",
       "4              3           0            0               0   \n",
       "\n",
       "  artist_composer_lyricist  song_lang_boolean  smaller_song  \\\n",
       "0                        0                  0             1   \n",
       "1                        0                  0             0   \n",
       "2                        0                  0             1   \n",
       "3                        0                  0             0   \n",
       "4                        0                  0             1   \n",
       "\n",
       "   count_song_played  count_artist_played  age_category  \n",
       "0                215                 1140           5.0  \n",
       "1                  1               303616           6.0  \n",
       "2                  4                  289           6.0  \n",
       "3                  1                    1           6.0  \n",
       "4                412                  427           5.0  \n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>msno</th>\n",
       "      <th>song_id</th>\n",
       "      <th>source_system_tab</th>\n",
       "      <th>source_screen_name</th>\n",
       "      <th>source_type</th>\n",
       "      <th>song_length</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>composer</th>\n",
       "      <th>...</th>\n",
       "      <th>composer_count</th>\n",
       "      <th>is_featured</th>\n",
       "      <th>artist_count</th>\n",
       "      <th>artist_composer</th>\n",
       "      <th>artist_composer_lyricist</th>\n",
       "      <th>song_lang_boolean</th>\n",
       "      <th>smaller_song</th>\n",
       "      <th>count_song_played</th>\n",
       "      <th>count_artist_played</th>\n",
       "      <th>age_category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=</td>\n",
       "      <td>WmHKgKMlp1lQMecNdNvDMkvIycZYHnFwDT72I5sIssc=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-library</td>\n",
       "      <td>224130</td>\n",
       "      <td>458</td>\n",
       "      <td>梁文音 (Rachel Liang)</td>\n",
       "      <td>Qi Zheng Zhang</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>694</td>\n",
       "      <td>13654</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=</td>\n",
       "      <td>y/rsZ9DC7FwK5F2PK2D5mj+aOBUJAjuu3dZ14NgE0vM=</td>\n",
       "      <td>my library</td>\n",
       "      <td>Local playlist more</td>\n",
       "      <td>local-library</td>\n",
       "      <td>320470</td>\n",
       "      <td>465</td>\n",
       "      <td>林俊傑 (JJ Lin)</td>\n",
       "      <td>林俊傑</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6090</td>\n",
       "      <td>115325</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>/uQAlrAkaczV+nWCd2sPF2ekvXPRipV7q0l+gbLuxjw=</td>\n",
       "      <td>8eZLFOdGVdXBSqoAv5nsLigeH2BvKXzTQYtUM53I0k4=</td>\n",
       "      <td>discover</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>song-based-playlist</td>\n",
       "      <td>315899</td>\n",
       "      <td>2022</td>\n",
       "      <td>Yu Takahashi (高橋優)</td>\n",
       "      <td>Yu Takahashi</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>989</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=</td>\n",
       "      <td>ztCf8thYsS4YN3GcIL/bvoxLm/T5mYBVKOO4C9NiVfQ=</td>\n",
       "      <td>radio</td>\n",
       "      <td>Radio</td>\n",
       "      <td>radio</td>\n",
       "      <td>285210</td>\n",
       "      <td>465</td>\n",
       "      <td>U2</td>\n",
       "      <td>The Edge| Adam Clayton| Larry Mullen| Jr.</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>31</td>\n",
       "      <td>698</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=</td>\n",
       "      <td>MKVMpslKcQhMaFEgcEQhEfi5+RZhMYlU3eRDpySrH8Y=</td>\n",
       "      <td>radio</td>\n",
       "      <td>Radio</td>\n",
       "      <td>radio</td>\n",
       "      <td>197590</td>\n",
       "      <td>873</td>\n",
       "      <td>Yoga Mr Sound</td>\n",
       "      <td>Neuromancer</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>180</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                          msno  \\\n",
       "0   0  V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=   \n",
       "1   1  V8ruy7SGk7tDm3zA51DPpn6qutt+vmKMBKa21dp54uM=   \n",
       "2   2  /uQAlrAkaczV+nWCd2sPF2ekvXPRipV7q0l+gbLuxjw=   \n",
       "3   3  1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=   \n",
       "4   4  1a6oo/iXKatxQx4eS9zTVD+KlSVaAFbTIqVvwLC1Y0k=   \n",
       "\n",
       "                                        song_id source_system_tab  \\\n",
       "0  WmHKgKMlp1lQMecNdNvDMkvIycZYHnFwDT72I5sIssc=        my library   \n",
       "1  y/rsZ9DC7FwK5F2PK2D5mj+aOBUJAjuu3dZ14NgE0vM=        my library   \n",
       "2  8eZLFOdGVdXBSqoAv5nsLigeH2BvKXzTQYtUM53I0k4=          discover   \n",
       "3  ztCf8thYsS4YN3GcIL/bvoxLm/T5mYBVKOO4C9NiVfQ=             radio   \n",
       "4  MKVMpslKcQhMaFEgcEQhEfi5+RZhMYlU3eRDpySrH8Y=             radio   \n",
       "\n",
       "    source_screen_name          source_type  song_length genre_ids  \\\n",
       "0  Local playlist more        local-library       224130       458   \n",
       "1  Local playlist more        local-library       320470       465   \n",
       "2              Unknown  song-based-playlist       315899      2022   \n",
       "3                Radio                radio       285210       465   \n",
       "4                Radio                radio       197590       873   \n",
       "\n",
       "          artist_name                                   composer  ...  \\\n",
       "0  梁文音 (Rachel Liang)                             Qi Zheng Zhang  ...   \n",
       "1        林俊傑 (JJ Lin)                                        林俊傑  ...   \n",
       "2  Yu Takahashi (高橋優)                               Yu Takahashi  ...   \n",
       "3                  U2  The Edge| Adam Clayton| Larry Mullen| Jr.  ...   \n",
       "4       Yoga Mr Sound                                Neuromancer  ...   \n",
       "\n",
       "  composer_count is_featured artist_count artist_composer  \\\n",
       "0              1           0            0               0   \n",
       "1              1           0            0               0   \n",
       "2              1           0            0               0   \n",
       "3              4           0            0               0   \n",
       "4              1           0            0               0   \n",
       "\n",
       "  artist_composer_lyricist  song_lang_boolean  smaller_song  \\\n",
       "0                        0                  0             1   \n",
       "1                        0                  0             0   \n",
       "2                        0                  1             0   \n",
       "3                        0                  0             0   \n",
       "4                        0                  0             1   \n",
       "\n",
       "   count_song_played  count_artist_played  age_category  \n",
       "0                694                13654           5.0  \n",
       "1               6090               115325           5.0  \n",
       "2                  5                  989           5.0  \n",
       "3                 31                  698           6.0  \n",
       "4                  5                  180           6.0  \n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#所有特征中是object类型的转换为类别型category"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "for col in train.columns:\n",
    "    if train[col].dtype == object:\n",
    "        train[col] = train[col].astype('category')\n",
    "        test[col] = test[col].astype('category')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "#保存特征数据\n",
    "train.to_csv(data_path +'FE_train_data_to_category.csv', index=False)\n",
    "test.to_csv(data_path +'FE_test_data_to_category.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
